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PREPACE 

During the 1970-71 school year the authors were part of a Rand team 
that made a detailed study of performance contracting programs in five 
school districts: Norfolk, Virginia; Texarkana, Arkansas; Gary, In- 
diana; Gilroy, California; and Grand Rapids, Michigan. This study was 
sponsored by the Assistant Secretary for Planning and Evaluation, U.S. 
Department of Health, Education, and Welfare, under Contract No. HEW- 
05-70-156. Prior to the 1971-72 school year the question arose as to 
what might happen to these programs, and the U.S. Office of Education 
agreed to sponsor a modest Rand reexamination. In the spring of 1972 
a visit was paid to each of the five districts. Interviews were con- 
ducted, classrooms were visited, and materials were collected to gain 
some understanding of how the 1970-71 programs had evolved. No attempt 
was made to replicate the earlier, detailed studies. The goal was merely 
to renew acquaintance with each project and see what had happened to 
them during the 1971-72 school year. This report summarizes the re- 
sults of our visits. 

The present investigation was sponsored by the U.S. Office of Ed- 
ucation pursuant to Contract No. OEC-0- 72-1381. 

P. Carpenter, A. W. Chalfant* G. R. Hall, M. L. Rapp, and G. C. 
Sumner, Case Studies in Educational Pei^fommae Contracting^ 6 Vols., 
The Rand Corporation, R-900/1-6-HEW, December 1971. 
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SUMKARY 



This study reports on visits in the spring of 1972 to five school 
districts that had Implemented educational performance contracting pro- 
grams during the 1970-71 school year. Some of the 1970-71 programs 
^^^^ continued, some had been cancelled, and some were "turnkeyed" 
^^^°P*=^^ ^o'^ In-house operations) for the 1971-72 school year. In two 
districts new programs were started In 1971-72. 
?! ^""^ review In the spring of 1972 generally reinforced the conclu- 

i slons we reached after a study of performance contracting programs dur- 

I ^"8 1970-71 school year. Changes In the content and methods of 

instruction, measurement and evaluation, and management seem to have 
I been brought about by the programs. The programs may not, however, be 

I having lasting Impacts on resource usage patterns. Achievement test 

I score results for some programs appear superior to gains achieved In 

1 conventional remedial programs, but In other programs the results are 

I same. The large gains hoped for by many proponents of per- 

I fonnance contracting have not usually been achieved. 

I In each of the districts a diagnostic-prescriptive approach to 

I Individualization of Instruction has bean encouraged by the performance 

I contracting programs. More attention Is being given to the need for 

I ^ ^^^^ of self-Instructional materials In such programs. Pro- 

f grams that had been convertea from performance contracts to In-house 

1 efforts tended to be adapted to Individual teaching styles, and more 

f diversity among classrooms resulted. 

I ^ The results obtained In the 1970-71 programs suggested that a re- 

I ductlon In the cost of remedial education might be achieved. However, 

|: ^ exception, the 1971-72 programs used about the same resource 

I support as they had the year before, or even greater support was pro- 

I vided. . 

I i'erformance contracting (which is still largely limited to reading 

y mathematics courses) ties contractor payments to student test re- 

f s"^*=s therefore focuses attention on testing and evaluation. The 

result has been increasing concern with development of better measure- 
ment techniques and evaluations. 
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The performance contracting programs forced school administrators 
to become Involved with the course of a project from the point of mak- 
ing contractual arrangements to the Implementation of the programs In 
the classrooms. This has had the effect of sensitizing administrators 
to problems and possibilities In areas such as new approaches to In- 
struction, teacher training, test and evaluation, union relationships, 
and acquisition and administration of federal funding. 

Only limited data on achievement gains during 1971-72 are avail- 
able. In Gary, the 22-day teacher strike led to cancellation of the 
post-test, so no gain scores are available. In Texarkana, the gains 
were uniformly less than the objectives; for example. Instead of 75 
percent of the students in the labs making one grade level gain or 
better, only 38 percent improved that much in reading comprehension 
and only 28 percent in arithmetic. In Norfolk the program group in 
most but not all cases did not show gains significantly different 
from the control group. In Grand Rapids the results were mixed; one 
program had gains of about two achievement years, and the others were 
in the 0.5 to 1.2 range. In Gilroy in a program started in 1971-72 only 
two of the 29 students achieved the target gain of nine months* growth. 
The students who had been in the 1970-71 program, but were not in such 
a program in 1971-72, did exceptionally well on the third-grade reading 
test. The fourth-grade results were no better than expected for those 
who had participated in the program, but nonparticipants made substan- 
tial gains. Whether these results were due to curriculum improvements 
cannot be determined. 
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I, XNXRODUCTION 



THE PRESENT STUDY 

Educational performance contracting was In vogue during the 1970-71 
school year and perhaps as many as 80 or 90 school districts experi- 
mented with the technique. Five of these districts were studied In 
detail by the present authors and their colleagues under the sporsor- 
ship of the U.S. Department of Health, Education, and Welfare.* In an 
effort to learn more about these programs, in the spring of 1972, un- 
der the sponsorship of the U.S. Office of Education, we visited each 
of the five districts and made a brief study of the current status of 
performance contracting. This report presents the conclusions drawn 
from these visits to Norfolk, Virginia; Grand Rapids, Michigan; Texar- 
kana, Arkansas; Gary, Indiana; and Gllroy, California. 

The various performance contracted programs studied in 1970-71 have 
been either (1) continued with very little modification, (2) continued 
on a fixed-fee rather than on a performance-payment basis, (3) "turn- 
keyed" with contractors supplying consultation support, or (4) discon- 
tinued. 

In the Jargon of perfornance contracting, a tumkeyed program re- 
fers to the adoption for operation by a school district of a system 
that a contractor has developed and has Implemented in the district, 
and that is running sufficiently well for the contractor merely to 
"turn over the key" to the school officials. Much of the literature 
on performance contracting has stressed the desirability of turnkeylng 
new programs, and this option has become of particular Interest. 

The disparate experiences and policies in the five districts, as 
well as the differences among the programs themselves, provided an op- 
portunity to study the evolution of performance contracting. At the 
same time, the diversity led us to treat each program individually. 
For each school district, we have tried to focus on the program elements 

5 

P. Carpenter, A. W. Chalfant, G. R. Hall, M. L. Rapp, and G. C. 
Sumner, Caee Studies in Educational Perfomcmae Contracting^ 6 Vols., 
The Rand Corporation, R-.900/1-6-HEW, DecenJber 1971. 
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and the policy issues that seemed most Important in that district. We 
believe that this approach has Increased the pertinence of our findings, 
although it results in an occasional unevenness m the discussion and 
analysis of the prograns. 

BACKGROUND OM PERFORMANCE CONTRACTING IN 1971-72 

During the 1970-71 academic year, many school districts sponsored 
performance contracts, professional education journals debated the con- 
cept, and the press and television reported extensively on the many 
projects. The 20-program experiment sponsored by the Office of Economic 
Opportunity was particularly in the public eye. Tne OEO ejiperlment had 
been planned as a one-year test and OEO held to Its plans and did not 
fund further programs for 1971-72. 

At the end of the 1^70-71 school year, several local evaluation 
reports were released including those for the Virginia, Texarkana-Edu- 
cational Development Laboratories, and Gllroy programs. None of these 
showed large gains In student achievement. At the start of the 1970-71 
year, performance contracting enthusiasts had talked about average achieve- 
ment gains on the order of 2 years In a single school year. Instead, 
the gains that were realized were of an order of magnitude familiar to 
compensatory education specialists -about 0.5 or 0.7, or even 1.0, years 
of gain per school year. In some cases the performance contracting gains 
were superior to gains of control groups, but in few cases were thev spec- 
tacularly large. 

The commercial firms, it appeared, have no magic solution for clos- 
ing the gap between the achievement test scores of students from advan- 
taged homes and those from disadvantaged backgrounds. It was also clear 
that performance contracting was not going to sweep the nation's schools. 
Indeed, a number of performance contracting firms left the field, some 
In serious financial difficulty. Many school officials shelved plans 
for performance contracting programs and turned to other proposals for 
educational change. A number of programs, however, were continued and 
some new ones were started. The Phi Delta Kappan estimated during 1971- 
72 that there were 100 performance contracts in effect; however, 69 of 
them were in Michigan where the State Department of Education has been 
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particularly Interested in the technique and has an unusual special 
program effect, 

E.aryone, however, waited for the results of the OEO experiment 
that was to provide definitive, rigorous experimental data on perfor- 
mance contracting. The results were long delayed and not released un- 
til February 1, 1972. OEO's assessment was devastatlngly sharp: 

The single most Important question for all concerned 
with the experiment Is: Was performance contracting more 
successful than traditional classroom methods In Improving 
the reading and math skills of poor children? The answer 
• • • is No* ' 

As might be expected. Individuals and groups who had opposed the 
experiment from the start responded with "I told you so." Many educa- 
tors who had been Interested In but uncertain about the concept shelved 
plans for performance contracting on the basis that the OEO results 
meant that "performance contracting was dead." However, a nunfcer of 
laymen, as well as professional educators and researchers, objected to 
these conclusions on the basis that they were premature. There have 
also been a number of attacks on OEO's experimental design and the va- 
lidity of Its statistical analysis and conclusions.* The controversy 
about performance contracting continued throughout the 1971-72 academic 
year and Into the 1972-73 school year. The focus of the controversy has 
shifted, however. Formerly, It was over the political and educational 



"OEO, Educators Debate Value of Performance Contracting." Phi 
Delta Kapipan, Vol. LIII, No. 7, March 1972, p. 451. 

An Experiment in Pevformanae Contracting: Swnmary of Preliminaru 
Results, OEO Pamphlet 3400-5, office of Economic Opportunity, Washing- 
ton, D.C. , February 1972, p. 17. 

Most of the criticisms have been made orally or In fugitive pub- 
lications. They are summarized, however. In J. A. Mecklenburger and 
D. M. Goldenbaum, "How OEO Failed Performance Contracting," Nation's 
Sohools, Vol. 89, No. 4, April 1972, pp. 31-32; C. Blaschke, "What OEO 
Did Not Say," Nation's Schools, Vol. 89, No. 5, May 1972, p. 36; and a 
statement by the contractors In Office of Economic Opportunity, An Ex- 
pertment %n Performance Contracting, OEO Pamphlet 3400-6, Washington 
D.C, June 1972, pp. 229-244. See also the references cited In Gary 
Saretsky, "The OEO P. c. Experiment and the John Henry Effect," Phi 
Delta Kappan, Vol. LIII, No. 9, May 1972, pp. 579-581. 
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desirability of contracting with private firms for instruction. Now it 
centers* around whether the 1970-71 results really indicated that per- 
formance contracting "failed." A major aspect of this debate is spec- 
ulation on the results after the initial iapleaentational difficulties 
encountered in all performance contracting programs had been solved. 
This controversy gives particular interest to our brief reexamination 
of the performance contracting programs studied In 1970-71. 
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II. NORFOUC. VIRGINIA 



INTRODUCTION 

During the 1970-71 school year, Norfolk participated in a state- 
wide program to improve reading skills through . . ance contract 
with Learning Research Associates (LRA).* LRA uw. guaranteed" that 
in one year the program would bring about gains of 1.7 grade levels in 
reading as measured by standardized achievement tests. The actual gains 
were considerably less than £his, as Table 1^ demonstrates. Note that 
in all but one case (the seventh grade), the treatment group gained 
somewhat less than the comparison groups. Whether the gaias of the 
treatment groups were significantly different from those of the comparl- 

Table 1 

COMPARISON OF GAIN SCORES IN 1970-71 PROGRAM 









Average 






Nunber c 


3rade-Level 


Grade Level 


Group Role 


Students 


Gain 


4 


Treatment 


49 


0.2 




Coiq>arison 


30 


0.5 


5 


Treatment 


44 


0.1 




Comparison 


30 


0.4 


6 


Treatment 


47 


0.1 




Comparison 


30 


0.3 


7 


Treatment 


51 


0.5 




Comparison 


28 


0.3 


8 


Treatment 


70 


0.7 


9 


Treatment 


39 


0.6 



The 1970-71 program was not continued as a formal demonstration but 
all 7 components were continued in one form or another as in-house pro- 
grams. See Charles Blaschke, "Experts Couple Warnings with Utest Find- 
ings," Nation's SohooU, Vol. 32, No. 6, January 1072, p. 25. 

From data presented to the Virginia State Board of Education by 
Q the State Department of Education on August 6, 1971. 
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son groups, we do not know. This information was not Included in the 
above report, and as of this writing the raw data have not been nade 
^v^-lable to the Norfolk City Schools by the evaluator, the Bureau of 
e .cational Research, Dhlversity of Virginia. 

LRA stood to n«ke a aaximum of $85 per student, or a total of about 
$21,250 ior the Norfolk portion of the Virginia program. Of this, 25 
percent was to be for student mastery of performance objectives. Thus, 
approximately $15,875 rode on the gain scores. Because of prorations 
for students who attended less than the required number of instructional 
periods or who were absent for one or acre of the testing sessions, 
LRA's final payment on the basis of gain scores was over 75 percent of 
the possible maximum; the payment on the basis of performance objectives 
was nearly equal to the maximum. 

The State Department of Education finished its report by recom- 
mending that the LRA program be continued in Virginia, with or without 
the direct involvement of the contractor. The reasons given for this 
support were the beneficial affective changes noted among the students 
and "the professional stimulation and interest generated among teachers 
by this instructional program."* 

The Norfolk program was exranded In 1971-72 from two schools (one 
elementary and one junior high) to ten schools (five elementary and 
five junior high), but not as a performance contract. The schools are 
listed in Table 2. 

Table 2 

NORFOLK SCHOOLS WITH HIGH INTENSITY CENTERS 

,» ^ Elementary Junior High 

1970-71 1971-72 1970-71 1971-72 

St. Helena St. Helena Jacox Jacox 
Madison Blair 
Monroe CaiH)08tella 
Stuart Ruffner 
Titus Willard 

Some 1250 students were involved in the 1971-72 program, in contrast to 
the 250 students in the 1970-71 program. 
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The 1971-72 program, termed the High Intensity Program, was an In- 
house effort by the Norfolk City Schools • LRA was paid through conven- 
tional contracting arrangements for furnishing only the materials for 
the Higih Intensity Centers • 

STUDENTS 

Norfolk undertook a massive busing program at the direction oi the 
U.S. Court during the 1971-72 school year. The objective was to pro- 
vide the same racial composition in every Norfolk public school by bus- 
ing whites or blacks or both. As one result, 3000 white students with- 
drew from the school system. 

During the 1970-71 school year, a number of the junior and senior 
high schools had been integrated with a consequent loss of white stu- 
dents at those levels. The following year, white enrollment was up in 
some of these schools. In fact, one teacher stated that Jacox Junior 
High had more white than black students in 1971-72. Ifost of the losses 
of white students occurred at the newly integrated schools. For exam- 
ple, Blair Junior High, which had an all white enrollment of around 
1400 in 1970-71, lost 400 to 500 students. 

In order to integrate the elementary schools with minimum disrup- 
tion in terms of student travel, most of them were made into lower ele- 
mentary schools comprising grades 1 through 4 or upper elementary schools 
coo^rising grades 5 and 6. St Helena Elementary was made into a lower 
elementary school for 1971-72. This meant that the school lost all of 
the students who had been in the 1970-71 program, which was provided 
for grades 4 throu^ 6. 

In principle, a student was a candidate for the program if his IQ 
was at least 75 and he had scored at least two years below grade level 
on the reading test given during the regular school testing program. 
These rules had to be modified in several cases. Because it was so 
small, for example, almost all of St. Helena*s third- and fourth-graders 
ended up in the program. Only the few who were reading at grade level 
were not included. 

The number of students chosen was dictated by the space available 
in each school. For example, at one school only a small conference room 
was available, and only 75 students could participate in the program there. 
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In the Junior high schools, students were selected Jointly by the 
reading teachers and the English teachers. At Blair Junior High, sev- 
enth-grade students had their schedules rearranged around reading as a 
core program. A few minutes were taken from each of their other sub- 
jects to make time for reading. Eighth- and ninth-grade students sub- 
stituted reading for English, social science, science, or another non- 
elective, and credit was given for the subject that was replaced. In 
the other Junior high schools, seventh-grade students were handled in 
the same way as those in the eighth and ninth grades. 

Teachers usually remarked that the students in the reading program 
came from the "modified" (i.e., slow) track, although there are offi- 
cially no tracks in Norfolk's schools. Only one teacher, however, sfated 
that she had a few students who should have been in special education 
but were put Into the reading program because of lack of space in spe- 
cial education classes. 

There were the usual problems with poor attendance and student 
tumovei One teacher had worked with over 200 students since the be- 
ginning of the year but had lost nearly 80 of them by spring. Over 
half of these withdrawals were for poor attendance or because the stu- 
dents would not work at their reading. On the plus side, slightly more 
than 40 percent of the withdrawals were because the teacher Judged the 
student had made sufficient improvement in reading. Another teacher 
stated that most of the absences in her class were due to suspensions. 

The only reading teacher who had been in the program both years 
remarked that her students in the second year were generally better than 
the first year and were not so far bfchlnd in reading. The other read- 
ing teachers all noted, however, tha'. many of their students could not 
read. The reading teacher gave each student a test to determine at 
what level of difficulty he should be working. The junior high teachers 
said their students were typically reading "at the third-grade level," 
which is the floor of the test administered. 

Can these students read? From the foregoing, it is apparent that 
the students in the reading program were severely deficient in reading 
skills as measured by standard instrtments. Yet Norfolk's Supervisor 
of English for Secondary Education stated that the students can read 
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more than they let on. The truth probably lies somewhere between 
these views. The students undoubtedly can "read" when it is in their 
interest to do so; for example, when they are looking at baseball scores 
or shopping for candy. This is not, however, the same reading measured 
by standard instruments— the ability to understand sentences and para- 
graphs. A contributing factor to students* lack of skills of these 
kinds will be discussed shortly. 

STAFF 

All of the teachers in the program were former reading teachers . 
In 1971 they participated in a summer workshop to become acquainted 
with the program. This workshop was conducted by Mr. Daniel Avent, the 
director of the program, and the two reading teachers from the 1970-71 
program. One of these was still teaching the second year; the other 
had moved on. 

We asked whether the program required any special skills on the 
part ot the teacher for its successful implementation. (It seeme<J pos- 
sible that the classroom management techniques required might be diffi- 
cult for some teachers.) The replies indicated that this was not the 
case. All of the required traits cited were those that one would ex- 
pect any good reading teacher to have—ability to communicate with the 
students, ability to establish rapport with them, knowledge of reading, 
a diagnostic point of view (ability to spot when a child is in trouble), 
a positive attitude, patience, and emotional stability. Of these, per- 
haps only the diagnostic point of view is one which levies special re- 
quirements, as most teachers would not have had experience with the 
diagrostic-prescriptive approach used in the program. 

Teacher training, which Avent cited as one of the problems of the 
program, should supply this. He noted that it took nearly the entire 
first semester for teachers new to the program to become confident in 
their ability to conduct it. Up to that point he had to hand-hold to 
a large extent. 

Avent had begun the program for 1971-72 without aides, although 
each of the teachers had had an aide the preceding year. It quickly 
became clear that aides were essential because of the individual 
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attention required. Therefore, Avent assigned an aide to each of the 
High Intensity Centers, At a nuinber of the junior high schools, an ad- 
ditional aide was assigned to the program by the principal (althou^ 
only one had been needed at the junior high level the year before) . 
This raises the question as to whether the program was drifting back to 
the resource-usage pattern of the former reading programs. 

INSTRUCTION 

The contrast between the Instructional methods used In the High In- 
tensity classrooms during 1970-71 and those used In the regular class- 
rooms was dramatic* The High Intensity teachers Implemented an open 
classroom concept, with each student working Individually, at his own 
pace, and responsible for managing his own time. In the regular class- 
rooms, the teachers directed almost all student activities. Students 
seemed to do almost no reading; Instead, they learned their lessons by 
listening to the teacher. In this way, they could be promoted year 
after year without ever having mastered any reading skills. 

During 1971-72, the reading teachers were no longer In the position 
of Implementing a program on which a contractor's payment depended and 
were no longer In direct contact with contractor personnel « They were 
therefore freer to Implement the program as they felt best, with the 
result that there were large observable differences In classroom prac- 
tice from teacher to teacher. One classroom, In particular, approached 
the atmosphere of a regular classroom, with teacher and aides responsi- 
ble for managing the students' work (although each student still had 
his own course of study). 

Whether this change had any effect on student achievement, we do 
not know, as the evaluation data have not been related to specific 
classrooms. 

MANAGEMENT 

Avent has continued to manage the program and act as liaison be* 
tween Alan Cohen (who worked for LRA on the design of the instructional 
program and is now with Random House) and Norfolk. Despite the appreci- 
able program development that went on last year, however, all of the 



reading teachers complained that they had had trouble getting materials. 
Statements of objectives, keys to the interim tests of objectives, and 
the tests themselves were not delivered in time, and some 'materials were 
still coming in April 1972. There seems to have been little excuse 
for this, since the decision to continue and expand the program had been 
made well before the close of the 1970-71 school year. 

Worse yet, deficiencies in the materials used in the program, which 
became clearly evident during the first year, continued throughout the 
1971-72 school year. These deficiencies may well have contributed to 
the very poor showing in gain scores the first year; Norfolk's failure 
to bring sufficient pressure on LRA and Cohen to remedy the problem is 
difficult to understand. 

MATERIALS AND EQUIPMENT 

The reading teachers noted the following deficiencies in the pro- 
gram materials: 

o Not enough materials were provided for teaching skills in read- 
ing comprehension at a low level of reading difficulty (ele- 
mentary level) . 

o Materials for teaching beginning reading skills (such as the 
Michigan Language Piogram and the First Talking Alphabet) were 
too babyish (Junior high level); students resent them. Yet, 
students need to learn these skills (such as relating the sound 
of a letter to its form). Materials for the mature nonreader 
(such as those produced by the U.S. Armed Forces Institute at 
Madison and the Job Corps) are needed. 

o There was too much emphasis on listening skills (Michigan Lan- 
guage Program), which students already have developed highly 
because the school situation demands it (Junior high level) . 

o Students were bored with the Science Research Associates (SRA) 
kits, which they have had since elementary school (Junior high 
level) . 

o Some materials teaching beginning skills (Durr-Hlllerich work 
on word attack, for example) are too difficult for the student 
population (Junior high level). 
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Despite these problems, reading teachers were generally pleased 
with the program materials. Several praised the variety available, 
\dilch made It possible to find something to fill almost any student's 
needs and to capture his Interest. There were materials to challenge 
the better readers, yet all students could find something they could 
cope with. Singled out for praise were books (paperbacks, Dr. Seuss, 
Bright and Early books, etc.), Barnell-Loft (word attack, comprehension). 
Random House Skllpacers (comprehension), and SRA We Are Black (compre- 
hension) . 

All of the teachers but one felt that the tape recorders were a 
distinct plus. Some teachers taped stories (accompanied by music) for 
students to listen to as they read them and gave each student his own 
tape to record stories on. One used them In conjunction with the Mich- 
igan Language Program and T>honlcs work. Another class had also worn 
out a Hoffman teaching machine left over from the previous reading pro- 
gram and had obtained another one. Kie teacher felt It supplied the 
privacy that some students needed* 

One teacher, however, disagreed with the majority view. She said 
that the directions given to the students by the t^ped programs were 
delivered too rapidly for the students to follow them. (She solved 
this problem by having aides Tead the direotione to the students 1) She 
also felt It was a waste of time for students to "read poetry Into the 
tape recorder," as had been suggested, when they needed to learn basic 
skills. She said that the primary problem was that the students stole 
the equipment as soon as they saw It. She therefore did not allow stu- 
dents to use the recorders. On this point, another teacher said that 
some tapes and books had been stolen, but she guessed that less than 
$200 worth of materials had been lost. 

PROGRAM TESTS 

This section will describe the tests that are integral to the read- 
ing program. Tests given as part of Norfolk's evaluation will be dis- 
cussed in the section on evaluation. 

For prescription of study programs. Random House has constructed 
a Check-In Test that will test up to 500 objectives and will cost $900 
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for a set for 125 students that will be used in the program. This was 
not available at the beginning of the 1971-72 year, so a Basic Test of 
Reading Comprehension (BTRC) was given to place each student in one of 
three "subsystems," sets of objectives and matched materials. Teachers 
felt that this was an accurate device, although one teacher systematic- 
ally placed students at lower levels than those indicated, to give them 
confidence. Another teacher felt that the BTRC should have tested more 
than comprehension. Teachers also used the Stanford I^agnostic Reading 
Test (DRT), vrtiich was used as part of the formal evaluation, to obtain 
further information on student weaknesses. 

As each student finished his work on an assigned rj)jective, the 
teacher was to give him a test ("check-out test") to see if he had mas- 
tered it. Some of these tests were not available in time, however; in- 
stead, teacher tallies of mastery of objectives were used as measures 
of student progress. Other tests were available without keys, and some 
test items were unreliable. On the whole, the teachers felt that these 
tests were adequate and thought the students actually enjoyed them. 

One teacher felt that too much time was spent on testing. Since 
she did not allow her students to have the answer keys for their class- 
room work, however, every assignment was like a test for them and the 
additional checkout tests were probably superfluous. 

EVALUATION 

For the evaluation, a control group was formed comprising five(\) 
students from each grade level at each school participating in the pro- 
gram. Students were to be randomly selected from among those who would 
have been selected for the program but had not been for lack of space. 

In November, program and control students were given the DRT Form 
X, and in December, a sample of students in each group was given the 
Otis-Lennon Test of Mental Ability. Form W of the DRT plus an attitude 
inventory developed at Norfolk were also administered in May. The DRT 
was chosen because it measures objectives taught in the program. The 
Otis-Lennon was chosen because It had been normed on the same groups 
as the DRT. 



So that they couldn't copy the answers. 
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By and large, the reading teachers found the DRT a helpful diagnos- 
tic instrument. Because it stresses vocabulary and word attack at the 
elementary level, elementary teachers expect a better showing on it than 
on the Metropolitan Achievement Test given last year. 

The Norfolk City schools' testing group prescribed Level II of the 
DRT for the junior high level because the content of Level I was so 
babyish that it would turn the students off. The result was that in 
one class almost all of the scores were at the bottom, and little infor- 
mation was provided to the reading teacher. Another teacher noted that 
some students were marking the tests at random because they were bored 
or because they couldn't read the questions. This underscores the lack 
of appropriate materials for the mature nonreader. 

TEACHER ATTITUDES 

Probably the firmest indicator of the attitude of the reading teach- 
ers toward the program is that all of them wanted to continue in it 
next year. In fact, one teacher was going to desert the school she had 
been teaching in to follow the program to another school. Teachers 
found it rewarding to have the materials they thought the students needed, 
and believed the students were learning. This was true for even the 
teacher who had regimented her classroom. Most of the teachers enjoyed 
the open classroom and the opportunity it gave them to adjust to the 
students' academic and emotional needs. 

STUDENT ATTITUDES 

The High Intensity classroom atmosphere seemed positive and purpose- 
ful. Students were generally orderly and in many classes asked to be 
allowed to work longer, even after the bell had rung to close the pe- 
riod. They may have been responding to the added freedom and variety 
of the open classroom rather than to the materials, but the whole effect 
was healthy. 

Teachers uniformly reported that students liked the program and 
were interested in the work. The students seemed actually to enjoy 
being kept busy at work they could handle and worked hard even in classes 
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where they got no grade in reading. Teachers believed students were 
more Interested, also, in their work in other classes and did more read- 
ing on their own. These attitude ohangee were seen as the primary ben-- 
efit from the program. An assistant principal thought that the program 
even ameliorated some behavior problems because of the individual atten- 
tion the students received. 

We asked whether students resented being in the "dumb" class. One 
teacher felt that all of her students did so. However, another felt 
that the poorer readers didn't mind being in the class (which was borne 
out by the reactions of some of the nonreaders) and pointed out that 
students were not forced to be in the class and could withdraw, if they 
wished, to make room for others who needed the instruction. The better 
readers, however, were sometimes resentful. In addition, some seventh- 
graders had no choice in the matt'^r. 

STUDENT COGNITIVE GAIN 

The Norfolk City Schools Department of Group Testing analyzed stu- 
dent cognitive gain in terms of differences in standard scores on pre- 
and post-administrations of the Stanford Diagnostic Reading Test. (Stan- 
dard scores are similar to raw scores except that they have been adjust- 
ed so that the mean of the national norm is 50 and the standard devia- 
tion is 10.) Level I was administered to students in grades 4, 5, and 
6, and Level II to students in grades 7, 8, and 9. 

The results are summarized in Tables 3 and 4. Table 3 presents 
pre-post comparisons for each of the subtests in the DRT Level I. Note 
that all of the standard score differences for the treatment group were 
significantly different from zero, whereas only the score for the blend'- 
ing subtest was significantly different from zet-) for the comparison 
group. The scores on begiming and ending sounds for the treatment and 
comparison groups are nearly the same, but the former is significant 
and the latter is not because of the large size of the treatment group 
and the small size of the comparison group. 



Eighth- and ninth-grade classes were graded because reading re- 
placed a subject required for graduation. 
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Table 3 



COMPARISON OF STANDARD SCORE DIFFERENCES FOR 
ELEMENTARY STUDENTS IN THE 1971-72 PROGRAM 
(Stanford Diagnostic Reading Test — Level I 
Grades A, 5, and 6) 



Subtest 


Treatment 
(397 students) 


Comparison 
(A5 students) 


Between-G 


Reading comprehension 


5.6A* 


1.96'' 


2.68* 


Vocabulary 


2.88* 


0.66^ 


1.52'' 


Auditory discrimination 


A. 11* 


2.39^ 


1.15'' 


Syllabication 


A.A7* 


2.60^ 


1.16^ 


Beginning and ending 
sounds 


2.60* 


i.73'' 


0.09^ 


Blending 


6.12* 


6. AO* 


0.22^ 


Sound discrimination 


A. 03* 


2.30^' 


1.26^' 


Composite 


A. 26* 


2.7a'' • 


1.07'' 



Significant at the 0.01 level. 

b 

Statistically not fignlflcant 

The betueen-group t shows that the two sets of differences differed 
significantly from one another only for reading comprehension. Thus, 
It Is still not possible to say that the LRA approach Is clearly more 
effective In teaching general reading skills. On the other hand, read- 
ing comprehension was the weakest area In the LRA program at the ele- 
mentary level during 1970-71 and Is probably the most Important single 
aspect of reading skill. Hence, the program has been Improved since 
that time. 

Table A presents similar data for students at the junior high level. 
The same general pattern seems to be repeated, although the significant 
differences between the two groupis arise In the areas of voodbulary and 
rate of reading. 



I 
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Table 4 

COMPARISON OF STANDARD SCORE DIFFERENCES FOR 
JUNIOR HIGH STUDENTS IN THE 1971-72 PROGRAM 
(Stanford Diagnostic Reading Test — Level II 
Grades 7, 8, and 9) 



Treatment Comparison 
Subtest (304 students) (38 students) Between-Group t 



Reading comprehension 


2.39* 


0.0^ 


1.43^ 


Vocabulary 


6.36* 


2.58^ 


2.04*^ 


Syllabication 


3.67* 


2.09^ 


0.81^ 


Sound discrimination 


2.3'^* 


2.85^ 


0.30^ 


Blending 


5.82* 


3.05^ 


1.84^ 


Rate of reading 


8.73* 


1.35^ 


3.70* 


Coiq>oslte 


4.88* 


1.99^ 


1.61^ 



ERIC 



significant at the 0.01 level. 

b 

Statistically not significant. 
^Significant at the 0.05 level. 



PLANS 

Norfolk plans to expand the program again for the 1972-73 school 
year. Five more centers will be Installed at the junior high level 
(Blair will have two centers) for a total of ten, and the five centers 
at the elementary level will be moved to the upper elementary schools 
comprising grades 5 and 6. Avent supported the Increased emphasis aa 
the upper grades (no students from grades 3 and 4 will he In tha pro- 
gram In 1972-73) with the argument that younger students require too 
much Individual assistance. 

Thus, the program was once again scheduled for expansion despite 
the lack of hard supportive data in the form of significantly better 
gain scores. When asked why, one person responded that it was because 
the Assistant Superintendent of Instruction likes the program and has 
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been sold on it from the beginning. This judgment seems unfair, even 
though it may be partly true. The obvious improvement in classroom 
ambience, the unanimous support of the program by the reading teachers, 
and the improve^ efficiency of resource use (even with the added aides 
at the junior hig^ level) can hardly be discounted, especially where 
there is no evidence that students are performing any worse than they 
did before. 

Next year several steps will. be taken to improve the materials at 
the junior high level. Only a part of the Michigan Language Program 
w-..:i be used and the Educational Development Laboratories study kits, 
which are beyond junior higji students, will be dropped. Saholastie 
books, presumably for elementary students, have been successful at the 
Junior high level and will be substituted for Landmark and Gateway ser- 
ies, which are too long, use difficult vocabulary, and require too much 
writing. Cohen has updated the program by incorporating an "audio read- 
ing progress laboratory" from the Educational Progress Center, more 
Random House materials, and more SRA materials for the junior high level. 
In spite of these additions, the materials will cost the same as pre- 
viously because of the deletions. 

The high cost of the Random House Check-In Test ($900 for a set for 
125 students) encct*raged Avent to use the DRT as a diagnostic instrument 
instead. The Sequential Test for Educational Progress will be used to 
measure cognitive gain. Another change will be the use of a larger com- 
parison group, v/hich will help improve the comparability of scores. 

CONCLUSIONS 

The 1970-71 performance contracting program with LRA has led to a 
considerably expanded remedial reading program in Norfolk. This pro- 
gram is based oa the materials and methods originally synthesized by 
LRA, particularly with regard to use of the diagnostic-prescriptive ap- 
proach for individualizing each student*s course of study. Teachers 
no longer have direct contact with LRA personnel, however, with the re- 
sult that there have been modifications of the program in general and 
in particular classrooms. One modification is significant In terms of 
the cost of the program— the use of two aides instead of one at the 
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junior high school level. The other has to do with the variations in 
classroom practice occasioned by differences in teaching style among 
indi-^idual teachers. Whether these variations have significantly af- 
fected the results of the program in terms of student test scores is 
unknown. 

Thus, one of the major effects of performance contracting may be 
that it is a means for assuring uniformity of teaching method and con- 
tent from classroom to classroom. The most significant effect of the 
program^ as far as staft and students are concerned, is the noticeable 
improvement it seems to have made in student attitudes toward reading. 
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III. TEXARKANA. ARKANSAS 



INTRODUCTION 

The pioneering Texarkana performance contracting program has evolved 
Into an In-house effort. In 1971-72 the Texarkana Dropout Prevention 
Program In Texarkana, Arkansas, and Llberty-Eylau, Texas (or as It Is 
called m Texarkana, the Title VIII program), drew funds from various 
sources In addition to Title VIII of the Elementary and Secondary Educa- 
tion Act. The 1971-72 program amalgamated money from Title I (teacher 
aides In the laboratories). Model Cities (elementary grades), and Title 
VIII (high school classes). 

The 1971-72 program consisted of six special reading and math labo- 
ratories, "basic classrooms" (formerly called turnkey classrooms), a 
vocational orientation class for sixth-grade students, and various an- 
cillary services Including curriculum development, In-servlce training, 
and community contact workers.* We shall discuss each major element 
briefly and then comment on the likely future of the program, the Im- 
pact of the former performance contracts on the 1971-72 program, and 
other Impacts on the Texarkana schools. 

THE 1971-72 PROGRAM 

Clinics a n d Laboratories 

There were nine special clinics or laboratories In operation serving 
484 students. The goal was to have approximately 100 students per cen- 
ter. The clinics could handle about 10 students per period and the lab- 
oratories about 20. The student assignment criteria were the same In 
1971-72 as In earlier years. The 1971-72 centers resembled the 1970-71 
centers under EDL cognizance. They used EDL machines and materials; how- 
ever, some materials that had been used In a consumable mode In 1970-/1 



* 



For a full description of the program, see L. H. Roberts and D. C. 
Andrew, The Texarkana Dropout Prevention Program^ Mid-Year Evaluation 
Rfort Region VIIl Education Setvlce Center. Magnolia, Ark., February 

\hese criteria were: IQ score of 75 or more and two or more grade 
levels deficient In reading and mathematics. 
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junior high school level. The other has to do with the variations in 
claBsroom practice occasioned by differences in teaching style among 
individual teachers. Whether these variations have significantly af- 
fected the results of the program in terns of student test scores is 
unknown* 

Thus, one of the major effects of performance contracting may be 
that it is a means for assuring uniformity of teaching method and con- 
tent from classroom to classroom. The most significant effect of the 
program, as far as staff and students are concerned, is the noticeable 
improvement it seems to have made in student attitudes toward reading. 
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III. TEXARKANA. ARKANSAS 



INTRODUCTION 

The pioneering Texarkana performance contracting program has evolved 
Into an In-house effort. In 1971-72 the Texarkana Dropout Prevention 
Program In Texarkana, Arkansas, and Llberty-Eylau, Texas (or as It Is 
called 1„ Texarkana, the Title VIII program) , drev funds from various 
sources in addition to Title VIII of the Elementary and Secondary Educa- 
tion Act. The 1971-72 program amalgamated money from Title I (teacher 
aides in the laboratories). Model Cities (elementary grades), and Title 
VIII (high school classes). 

The 1971-72 program consisted of six special reading and math labo- 
ratories, "basic classrooms" (formerly called turnkey classrooms), a 
vocational orientation class for sixth-grade students, and various an- 
cillary services including curriculum development, in-service training, 
and community contact workers.* We shall discuss each major element 
briefly and then comment on the likely future of the program, the im- 
pact of the former performance contracts on the 1971-72 program, and 
other impacts on the Texarkana schools. 

THE 1971-72 PROGRAM 

Clinics and Laboratories 

There were nine special clinics or laboratories in operation serving 
484 students. The goal was to have approximately 100 students per cen- 
ter. The clinics could handle about 10 students per period and the lab- 
oratories about 20. The student assignment criteria were the same in 
1971-72 as in earlier years. + The 1971-72 centers resembled the 1970-71 
centers under EDL cognizance. They used EDL machines and materials; how- 
ever, some materials that had been used in a consumable mode in 1970-71 



A A description of the program, see L. H. Roberts and D. C. 

ttlZ' Tfoilfm^lJ^°^T* Prevention Program, Mid-Year Evaluation 
29^972 Education Service Center, Magnolia, Ark., February 

^These criteria were: IQ score of 75 or more and two or more grade 
levels deficient in reading and mathematics. 
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were adapted to be nonconsumable . EDL was not Involved In l<371-72 except 
as a sourca of materials purchases for replacement purposes. 

The junior high and high school laboratories were primarily reme- 
dial, while the elementary clinics Involved more diagnostic activities. 
In particular, the clinics have been Interested In detecting and assist- 
ing children with perceptual difficulties. The clinic at the Washing- 
ton Sixth Grade School has a different function; It enrolls children 
who do not qualify for the Rapid Learning Center because their IQ test 
scores fall below the cutoff point. 

In comparing the 1970-71 and 1971-72 centers, two major changes 
stand out. First. In 1971-72 a wider variety of materials was used. 
Second, there was less emphasis on the teachers and students covering 
a set program of material In a set period of time. Teachers made more 
u^e of movies, newspapers, and other materials and slowed down the pace 
or repeated material If they desired. The teachers who were Interviewed 
favored both of these changes because they felt under less pressure. 

Basic Classrooms 

Texarkana uses a 3- and sometimes 4-track curriculum system. Orig- 
inally, the three tracks were called advanced, regular, and basic. In 
1971-72 the basic track In the schools Involved In the Title VIII proj- 
ect was replaced by "basic classrooms." These classrooms also replaced 
the turnkey classrooms used In the 1970-71 program. There were 21 basic 
classroom teachers. During the summer of 1971 there were several work- 
shops that displayed materials from different manufacturers. The teach- | 
ers selected materials and were assigned aides to help prepare them for j 

use in their classrooms. j 

i 

The basic classrooms that are part of the Title VIII program have 1 

supplementary types of materials that were not available before. This i 
seems to be the major difference from last year. Only in isolated cases 

are Dorsett materials being used, and then only by the special education j 

teachers and occasionally in adult education. Use of EDL materials and • 

procedures is largely limited to the laboratories. | 

The students in the basic classrooms had not necessarily graduated j 

from the laboratories or clinics. Nor had the basic classroom teachers | 

been involved with the Dorsett or FDL programs. In fact, only one of the j 



ERXC teachers has had much contact with the performance contracting programs. 
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Vocatlonal Orientation 

Students ar the Washington Sixth Grade School receive 12 weeks of 
vocational orientation Instruction as part of the 36-week social studies 
program. One-third of the student body is rotated through the program 
at a time. The goal is to Increase the students' occupational aware- 
ness before entering Junior high. The school system has decided that 
the usual high school vocational orientation programs need supplementa- 
tion, and attempts to Improve the students' information and attitudes 
toward careers must be made earlier in their schooling. 

Curriculum Development 

During 1971-72, representatives of the schools composing what Tex- 
arkana calls its Title VIII "model subsystem"-College Hill Elementary, 
Washington Sixth Grade. College Hill Junior High, and Arkansas High- ' 
formed a curriculum study committee, whose task was to produce a cur- 
riculum guide for individualized Instruction in math and language arts 
for grades 1 through 12. The teachers met each Monday evening for three 
hours and received a stipend for participating. The basic accomplishment 
of the committee was a scope and sequence objectives keyed for each grade 
level for each of three ability tracks. The committee had hoped to de- 
velop guidelines for choosing methods to diagnose student needs and to 
prescribe materials and instructional methods, but time did not permit. 
The Texarkana administration takes great pride in the accomplishments 
of the committee and believes that its efforts have been well received 
by the teaching staff.* We noted, however, from a questionnaire submit- 
ted to the teachers who had reviewed the committee's report that a few 
felt that the objectives were too ambitious for their students. 

In-Service Teacher Training 

Training consisted of a program of workshop sessions in Indfvid- 
ualized Instruction and behavioral modification techniques . Occasional 



erJc 



The evaluators reported that 98 percent of the 170 reviewers felt 
that the committee s results gave. "Excellent" and "Adequate" ratines 

^ J °* ^* Texarkana Dropout Prevention Proaram 

Thtrd Operational Year, 19?U?2, Region VIII Education ServJcHenJ^?! ' 
^gnolia, Arkansas, July 15, 1972, p. 95. (Hereafter cited as Hnal 
Evaluation Report^ 1972-78.) 
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guest lecturers were Invited. Teachers who attended the sessions re- 
ceived a stipend. 

Community Contact Workers 

There were 11 home contact workers assigned to the project. They 
visited the homes of students who were not attending school or showed 
signs of other problems. The contact workers also made periodic visits 
to the homes of students who did not show signs of difficulty. This 
component of the program was established because students with academic 
difficulties often have special home problems. 

REACTIONS TO 1971-72 PROGRAM 

In comparison with the 1970-71 program, one laboratory teacher felt 
that In 1971-72 there was less pressure to cover a specified amount of 
material In a specified period of time. In 1971*72 the students In the 
laboratories spent three days of the week working on t' j EDL materials 
used the previous year as well as SRA materials that had been added to 
the curriculum. During the other two days they engaged In directed or 
free reading and occasionally, about once a month, attended movies spon- 
sored by the high school English Department. The teacher also expressed 
a feeling of being less ostracized by the other teachers. Apparently 
In 1970-71 he had been the object of some hostility. 

Another teacher expressed even stronger opinions. She stated that 
she much preferred the 1971-72 program to the EDL program. She said 
she *'hated" performance contracting because of the pressure that It 
placed on the teacher. When asked for more details, she answered that 
In 1970-71 the component manager would announce that In two weeks' time 
he would test, say, cycles 10 through 20 of the EDL material; she would 
therefore have to rush through the cycles. During 1971-72 she could go 
back and redo cycles If she felt they were unsatisfactory, or not worry 
about how fast she was getting through them, fb^s helps the classroom 
too, she said, because when a teacher Is unde..' ^^essure the children 
feel It. She said she "loves" the SRA materials and the children "love" 
the SRA materials because they can respond; in the regular classes they 
cannot respond because they are always behind. She dislikes the Iowa 
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Test of Basic Skills (ITBS) because. In her judgment, It does not accu- 
rately measure what the students are doing. She also criticized the 
testing procedure. In 1970-71, she stated, the examiners herded the stu- 
dents to the cafeteria and administered the tests to a large group. She 
believes that slow learners, such as the students In the laboratories, 
do not understand directions unless they are repeated a number of times 
and, thus, the evaluation did not measure what was being learned. She 
is using the Hotel Test for evaluation In accordance with Texarkana school 
practice; but she Is also using some other evaluation Instruments to get 
further Information she wants. She used the Sullivan Placement Test 
for arithmetic when the students entered her laboratory. She gave the 
same placement test at the end of the program and computed where the stu- 
dents would be were they to have entered the program In May. The com- 
puted difference Is her own measure of success. The teacher noted a 
positive change In the attitude and manners of the children. 

Another teacher told us that her group reads the newspaper You and 
your World every Friday. Use of this newspaper started In 1971-72 to 
break up the "cycling," as the teachers and students call the EDL ma- 
terial. The teacher explained the lack of achievement score gains on 
the ITBS as due to the length of the test; she says that her students 
have a mental block about long tests. There was not much difference 
between the 1971-72 program and EDL's program the year before, she said, 
except that math and reading are no longer combined. She stated that 
there were less discipline problems this year. She also said that the 
other teachers had been jealous of her small class and the new material. 

An Interesting perspective on the 1971-72 program was provided by 
Robert E. Kraner of EPIC Diversified Systems Corporation of Tucson, 
Arizona, the educational auditor for the program. In Kraner's opinion, 
the 1971-72 evaluation design was good and the design for the forthcoming 
1972-73 year was excellent. He feels that It takes about three years 
for a program to develop an efficient evaluation. The key element. In 
his view. Is Internalized objectives. In the first year of a program, 
objectives delineated In the evaluation design are not likely to match 
the objectives conceived by the operating personnel. In the 1971-72 
Texarkana program. In contrast, Kraner stated, '•You can look at the 
O objectives in the evaluation plan and compare them with the files maln- 

operating personnel, and you find a very high match." 
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Kraner regards this as evidence that the evaluation activities match 
the program activities. He also stated that It made it easier to audit 
the program because one could merely check files and records and the 
material was either there or not, obviating an elaborate translation 
from project data on evaluation data. 

Kraner stated that the testing he observed in 1971-72 had been the 
best Job of teacher-administered testing he had seen. He is generally 
a defender of using achievement testing in evaluation. In addition to 
the ITBS, extensive use is being made of the Betel Test. It has become 
the prime diagnostic and evaluation Instrument for reading for the Title 
VIII program. Kraner summed up how the 1971-72 program has changed from 
earlier programs with three points: First, the evaluators are obtaining 
relevant data; second, testing has Improved; and third, there is more 
communication between evaluation and program Improvement. He stated 
that there is more understanding by the teachers of how they are being 
evaluated, and that a lack of such understanding accounts for much of 
the apparent failure of Title I programs. 

We askea about the 1971-72 evaluation activities of the Region 
VIII Center in Magnolia. Kraner stated, "It is doing less and less, 
which is good." The evaluator in this kind of program, he said, should 
work himself out of a job. The evaluators are less Involved in imple- 
menting the program and project personnel are more autonomous, which 
he regards as a sign of success. 

epic's role will not change basically in future years. However, 
the program is now much easier to audit. Kraner stated that he can 
now do in one day what used to take several days, because he can go 
through the files and get the data for easy spot checks. 

It 

Program Outcomes 

The 1971-72 program did not achieve its stated objectives. We will 
not repeat all the 20 major findings'*" but merely list four outcomes that 
llJ.ustrate the results: 



We discuss only what the evaluators call "product outcomes." For 
^077*^*^^°" process evaluation finding see the Final Report 
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Ibid, pp. 93-96. 



-26- 



o One objective was to decrease the dropout rate. In- 
stead the rate Increased slightly from 5.44 percent 
in 1970-71 to 6.80 percent in 1971-72. 

o Attendance did not Increase but decreased from 94.2 
percent in 1970-71 to 93.4 percent in 1971-72. At- 
tendance of program participants was slightly worse 
than the average attendance at all target schools. 

o One objective for the labs was to assist 75 percent 
of the students to make one grade level gain or more. 
Actual percentages of students making such a gain 
were: 

— Reading comprehension 38 

— Arithmetic 28 

These figures represented increases over 1970-71 results of 14 
centage points for reading and 4 percentage points for arithmetic. 

o The objective for the basic English classes was 

one grade level or more gain for 75 percent of the 
students. Actual percentages of students making 
such a gain were; 

— Reading comprehension 41 

— Arithmetic 31 

The evaluators concluded that: 

Since none of the expected outcomes for the overall 
program was achieved, it appears that the standards 
of acceptable performance set by the objectives were 
not realistic, or the dropout prevention program 
seems to have little effect on the total school p- o- 
gram. 



*Ibid, p. 98. 



The associated recoicmendatlon was: 

It is recommended that in the 1972-73 program the 
standards of acceptable performance expressed In 
each objective be reasonable and realistic, based 
upon baseline data from previous years* experiences 
and the judgment of teachers In the program.* 

It should be noted that the objectives were either some absolute 
standard — such as 75 percent of the students making oae grade level gain 
or better — or defined In Improvements from the previous year. This eval- 
uation approach leaves unanswered the question of how the outcome of 
the program compared with the probable results In the absence of a pro- 
gram. To deal with this Issue would have required a cross-sectional 
control group which was not used In the Texarkana program. How one sets 
a reasonable set of standards In the absence of some control group is 
an extremely difficult problem as the Texarkana experience Indicates. 
Nonetheless, It Is clear that In terms of the Impact on achievement 
test scores the first three years of the program have failed to make 
the substantial Impacts that had been hoped for. 

THE FUTURE OF THE PROGRAM 

The Texarkana Dropout Prevention Program had been Initially planned 
a. a five-year effort costing $1.5 million. The planned time phasing 
was $27C thousand for the first year, $311 thousand for the second, a 
peak of $420 thousand In the third year, and a leveling off to $290 
thousand for the fourth and fifth years. Because 1973-74 will be the 
fifth and under present arrangement the last year, the future of the 
program Is In question. We explored the likelihood that program expen- 
ses might be picked up by the local school boards. Cognizant officials 
think that this would be unlikely. Parts of the project could be continued 
since the hardware has been paid for and the software costs are not large. 
However, even a bare-bones program would cost at least $150 thousand a 
year. This Is equivalent to five mills on the Texarkana, Arkansas, tax 
rate. Officials do not believe that the taxpayers would approve such an 
increase for a special instructional program. 



*Ibid, p. 97. 
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Despite their pessimism about the likelihood of local financing 
for the program, officials are optimistic about continuing the various 
activities. They believe that various federal or state programs will 
provide an opportunity not only to continue but also even expand some 
of this year's efforts. The current view Is that any educational In- 
novation that looks promising must be attempted, even If there Is no 
guarantee of long-term financing, and If the program works out the 
financing will probably work Itself out. There Is no expectation, how- 
ever, that the number of laboratories or clinics will expand. Nor is 
there expectation that the materials and techniques used by the perfor- 
mance contractors will be adopted widely in standard classrooms. 

PERFORMANCE CONTRACTING AND EDUCATIONAL CHANGE 

Texarkana officials unanimously Insist that former performance con- 
tracts served as change agents leading to a number of innovations in 
the Texarkana schools. They argue that, first, performance contracting 
has changed thinking of teachers and officials; second, it has developed 
administrative skills useful in other projects; and third, it has led 
to a number of improvements in Texarkana's educational program. We pur- 
sued this subject in some detail because the technology and techniques 
of the performance contractors have not been widely replicated or adopt- 
ed in Texarkana classrooms and also because the 1971-72 program did not 
seem to be unusual compared with compensatory education programs else- 
where • 

Several administrators and teachers offered explanations of the 
relationship between performance contracting and change. First, per- 
f ormar • i contracting has led teachers to be less conservative about edu- 
cational change. The Title VIII program forced them to perceive that 
there v'ere other ways to teach besides the old methods they were using. 
In addition, the program has allowed Texarkana to upgrade its in-service 
training. The result of this conbination is that as teachers are exposed 
to nuw ideas, they become more receptive to educational innovations. An 
administrator stated that this benefit was the number one achievement of 
the performance contracts. 
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One of the teachers told us that during the last 12 years the Tex- 
arkana schools have had to deal with a greater variety of needs than 
ever before, and she felt that the Title VIII program had reinforced 
the teachers by providing more materials and instruction. She also 
emphasized that teachers were more receptive to new materials because 
of the performance contracts; they felt less threatened by new mate- 
rials since they had seen others using them. Teachers also tend to be 
more interested if they see other people making changes. 

The second explanation of the way that the program hp.d been a 
change agent was that it had forced administrators to think in system 
terms. One administrator said that when he had been a principal he 
Just went aliead and did things as he thought of them. The Title VIII 
program had forced him and other administrators to articulate their 
objectives on developing plans to obtain the teachers and other re- 
sources required and organizing the resources into a coherent project. 
Moreover, the fact that the program was evaluated meant that someone 
was "holding his feet to the fire." He felt that this had been a valu- 
able experience. Superintendent Trice independently made a similar 
point. He stated that the Title VIII program had given Texarkana the 
experience not only to obtain funding for other special programs but 
also to be able to manage these programs. 

The question naturally arises of why one needs performance con- 
tracting or Title VIII funds to acconqplish these changes. Wouldn*t 
any special program do as well? This question was addressed to an ad- 
ministrator who argued that Texarkana would not have made the changes 
without the Title VIII funds and the performance contracting. He main- 
tained that the Dropout Prevention Program had led to an improvement 
in the total school program, citing as examples the curriculum revision, 
the basic classrooms, and in-service education. He stated that Tex- 
arkana, Arkansas, was spending about $500 per child, whereas Texarkana, 
Texas, was spending about $700 per child. Arkansas has to compete with 
Texas for teachers and therefore the Arkansas schools find it difficult 
to finance Improvements as in-service education except through federal 
funds. He stated that the school could not have had the basic classes 
and the curriculum revision without Title VIII. First, he explained, 

ERiC 
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In Texarkana they had had bette* leadership In the Title VIII program 
than In the usual school program because It was a special program with 
specialized leadership requirements. The most Important thing, he said. 
Is that Title I has become so hlgjily categorical that It Is hard to live 
under the Arkansas guidelines and still have any kind of Innovative 
program. For example, when the poorest child In school comes to class 
barefoot on the coldett day It Is not possible to buy him shoes unless 
he has a teacher funded by Title I. Title VIII has a broader perspec- 
tive, he continued, and the model subsystem therefore could be more In- 
novative. Also, under Title I It Is hard to plan ahead because of 
changing laws and guidelines, whereas In the Dropout Prevention Program 
Texarkana could plan more effectively. 

The administrator's basic position Is that Title VIII programs were 
sufficiently flexible to permit a major change In procedures such as 
performance contracting and this. In turn, had led to a number of ancil- 
lary activities that were having system-wide impacts through curriculum 
changes. In-service training, and other services. It still remains un- 
clear, however, why performance contracting was t Important a factor 
In providing administrative flexibility as the Texarkana officials feel 
that It was. 

Superintendent Trice told us that he felt the performance contract- 
ing program had been a great achievement. Everyone In the system was 
more creative and Innovative because of It. He also felt that Texarkana 
had been successful In keeping children In the school. Martin Fllogamo, 
the Project Director, thought that Texarkana was not going to see much 
further Improvement on this score. Once down to the current 4-percent 
dropout rate, he questioned whether you could go any further. 

When Trice was pressed on what evidence there was that performance 
contracting had led to a change In the teachers, he cited the Follow 
Through Program In which Texarkana participated. He stated that at one 
time teachers vould have been very resentful of such a new program. 
But when the possibility arose to participate in Follow Through, teach- 
ers were very Interested, and « group was sent to three states to ex- 
amine programs. The teachers selected the Pittsburg program. As an- 
other example. Trice cited the Title III program in which Texarkana is 
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carrePtly attempting to develop an Innovative program Involving career 
education as early, perhaps, as the third grade. He argues that the 
teachers have a changed attitude toward such programs. 

SUMMARY 

The 1971-72 Dropout Prevention Program Involves special reading 
and mathematics clinics and laboratories, new materials and support for 
students m the lowest ability track, In-servlce training, and other 
services. The project Is the result of much effort by. the Texarkana 
schools and represents a significant Improvement In their program. Tex- 
arkana rightfully takes pride in the innovative character of the program 
and its content. 

It is useful, however, to distinguish between the content of ? pro- 
gram and the way it is organii^ed, that is, the manner in which the edu 
catlonal services are delivered.* In 1969-70, and 1970-71 when the con 
tracts with Dorsett and EDL were in effect, the TexarKana program in- 
volved a very unconventional way of organizing educational programs as 
well as atypical material and Instructional techniques. In 1971-72 the 
content of the program was still designed to Include new materials and 
techniques, however, the "delivery system" or program organization was 
similar to that v id in many special school programs. 

Texarkana officials support the proposition that perfont^nce con- 
tracting was a change agent. The/ cite various examples, including a 
curriculum revision program funded by Title VIII, in-service training 
programs, and participation in other programs such as Follow Through. 
They argue that these efforts would not have been undertaken had not 
teachers become more aware of the opportunities for change and the need 
for change, thereby becoming less conservative and more open to innova- 
tion. These changes, they believe, were a concomitant of the perfor- 
mance contracting program. Title VIII, they state, provides a flexibil- 
ity to experiment that is not present with Title I funds and similar 
funding sources . 



This distinction has been urged by Dennis Young of the Urban Insti- 
tute, Washington, D.C. , in Evaluation of Organizational Change: The Case 
of Perfomanoe Contvaoting, a paper prepared for the 28th Congress of the 
^ International Institute ox Public Finance, New York, Septenfcer 12-15, 1972. 
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When performance contracting was first Introduced into education 
with the state of the Texarkana program there was much discussion about 
Its potential as a change agent. One of the hopes at that time was that 
it would be a way of Introducing Into school systems Integrated systems 
developed by contractors. This has not happened In Texarkana. The 
changes cited above are not extensions of the Dorsett or EDL systems. 
The EDL system remains, with some modifications. In operation In the' 
labs and clinics but further applications are unlikely. The Dorsett 
system Is receiving only Isolated use In Texarkana. There appear to 
be no plans to replicate or appl, broadly the contractor systems. Tex- 
arkana would -like to continue the labs and other components of the pres- 
ent program when the Title VIII money and other funds now supporting 
the project run out. Such continuation, however, appears dependent on 
federal or state support. The odds for support from general school funds 
are low* 

The Texarkana evidence is that performance contracting has not been 
a change agent in the sense that it has led to widespread tumkeying of 
contractor developed technology. It has been, however, a change agent 
by providing Texarkana with an opportunity to interest teachers in edu- 
cational Innovation and providing Texarkana with an opportunity for 
various experiments. 



-33- 



IV. GARY. INDIANA 



THE 1971-72 PROGRAM 

Banneker School opened In September 1971 with 707 pupils and 28 cer- 
tificated teachers, with this pup-» Vteacher ratio, the average class 
size fell within the terms of the collective bargaining agreement be- 
tween School City (the Gary public schools) and the Gary Teachers Un- 
ion, but the program Involved significantly more than the 21 or 22 
teachers with which Behavioral Research Laboratories (BRL) had started 
the prior year and about 100 fewer students. The 1971-72 program must 
have been more expensive for BRL than had been anticipated In 1970. 
This assumption Is reinforced If one assumes that only about half of 
the students or less will qualify for achievement payments. This ap- 
pears to be a more realistic assumption than the 100-percent figure 
announced as a program objective. 

Enrollment dropped from the approximately 800 enrolled In 1970-71 

to 707 students In 1971-72, 530 from the Banneker area and 177 from 
it 

outside. The contract calls for a minimum enrollment of 700 and speaks 
of an "assumed" enrollment of 800. Apparently there are several reasons 
for the decline. School enrollment is down throughout Gary. Decline 
in steel demand raised the unemployment rate in Gary during some pe- 
riods in 1971-72 to 53 percert, leading to some outmigration. Demo- 
graphic changes have reduced the school population around Banneker.* 
Most of the transfers from the program have been because of family moves. 
It appears that much of the decline in enrollment reflects changes not 
related to the Banneker program. However, it is obvious that the program 



it 

Charles Young, Jr.. "Attitudes of the Banneker School Community in 
Gary, Indiana, Toward a Performance Contract Program," Ed.D. thesis 
submitted to the Graduate College of the University of Illinois at 
Urbana, Champaign, Urbana, Illinois, 1972, p. 108 (unpublished), 
f 

Young states that the Banneker attendance area consists of two 
parts. The northern part is mostly populated by middle-income residents 
with small families and has a low turnover rate. The southern part of 
the area consists of low-inoome families, many living on farms. Popu- 
lation in this area is falling as better housing is opening up in other 
parts of Gary. (Ibid, pp. 104-109.) 
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Is not appealing to enough Gary parents to draw sufficient students to 
fill the available seats. Young estimates that around 245 children 
living In the Banneker attendance area do not attend that school.* 

Gary's financial crisis and a 22-day teachers' strike In the late 
spring of 1972, as well as the general decline In Interest In performance 
contracting, moved Banneker away from the spotlight of publicity it 
had received during 1970-71. Nonetheless, according to Brian Fitch, the 
BRL Center Manager, the school had about 6500 visitors In 1971-72. Tues- 
days and Thursdays were designated as visitor days, and each week two 
teachers were selected to host classroom visits. 

Banneker continued to devote more time to reading and mathematics 
than Is typical in Gary or other Indiana schools. The extra time was 
taken from the enrichment areas (music, art, and physical education). 

The 1971-72 program differed from the 1970-71 program in two pro- 
cedural respects: The children were not transferred from classroom to 
classroom as often as they were in 1970-71. The problem was that the 
children tended to get stacked up at the top levels, and teachers did 
not want to transfer into new classrooms. As a result, in 1971-72 
students stayed in the same classroom all year and new materials were 
provided to them as they progressed. This represents a wjor departure 
from the theory of the Inltlil program. There was also a minor change: 
The 20-minute module was not used, on the basis that with a more 
Individualized program ^t w-'is not needed. 

A very Important difference between the 1970-71 and 1971-72 pro- 
grams was in school governance. As noted in our earlier study, one 
unique feature of the 1970-71 program was the role played by the five 
master teachers in directing the school During 1971-72 the new prin- 
cipal played a more traditional role as school administrator, and it 
follows that the role of the master teachers was more conventional. 

The committee structure was revised. Instead of the curriculum 
managers forming one major committee with a broad cognizance over 



*Ibld, p. 108. 
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school affairs, a variety of specialized committees was used, each com- 
posed of a curriculum manager and other teachers. Actually, It Is Im- 
precise to use the term curriculum manager, since In 1971-72 these 
teachers were designated as Department Heads. There were five depart- 
ments: mathematics, social studies, science, language arts, and en- 
rlchment . 

Differences of opinion on questions of discipline between BRL and 
parents appear to have been resolved. The current policy bans corporal 
punishment and encourages a humane atmosphere, but there Is no attempt 
to obtain a highly permissive atmosphere, to which parents object. The 
program has been less successful In resolving differences between BRL 
and the teachers. The Gary Teachers Union remains critical of the 
program, but had many higher-priority concerns In 19/1-72 and therefore 
had little to do with Banneker. Due to the drop In enrollment, a reduc- 
tion In teaching staff would have been possible. The criteria that BRL 
wanted to use to select teachers for a reduction In force conflicted 
with the School City-Gary Teachers Union contract. After negotiations 
among the union. School City, and BRL. the matter was settled by cancel- 
ling the proposed reduction and retaining all teachers. The conflict, 
however, left some hard feelings at Banneker. 

In general, the teaching staff remains divided about the merits of 
the BRL program. There Is, however, faculty feeling that the BRL and 
School City administrations have not been sufficiently solicitous about, 
and responsive to, teacher views and suggestions. 

The most significant change between 1970-71 and 1971-72 was the 
stress on curriculum development. In the spring of 1972 Brian Fitch 
emphasized to us that he believed the development of new and superior 
teaching materials was the essence of the program. This belief was re- 
flected In the 1971-72 activities at Banneker. 

BRL concentrated on program design and Implementation In language 
arts and mathematics. Fitch Indicated that the Sullivan materials used 
In 1970-71 are primarily oriented toward reading Instruction and that 
BRL wanted an Individualized total language arts program Including an 
oral component, spelling, and handwriting. In the spring of 1971, BRL 
asked a group of consultants to write the objectives for such a program. 



-36- 



During the summer, 18 Banneker teachers worked to specify the learning 
activities to teach uhe different objectives. In language arts and 
math, the BRL objective was total Individualization of the program. 

In social studies In 1971-72 Banneker used the Allyn and Bacon ma- 
terials. The goal was to develop a good format for small group Instruc- 
tion and to develop materials to supplement the Allyn and Bacon series. 
In science Banneker used the AAAS material, without addition or change, 
because BRL believed the materials were good and the teachers had been 
trained In their use. Fitch stated Banneker has not done very much with 
the enrichment areas, which are presented In a rather traditional way. 

The language arts materials are being marketed nationally by BRL. 
School City will receive a 2-percent royalty. The development of math- 
ematics materials Is not as far along, but It also Is nearlng a point 
where BRL will market the system, paying royalties to the Gary schools. 
The superintendent hopes to develop a trust fund arrangement so that 
the royalties can be used for Improvements at Banneker. 

Concern about BRL*s marketing plans was expressed to us. Some ob- 
servers wondered whether the materials had been tested sufficiently and 
whether they might apply too specifically to the Banneker situation. 
Obviously BRL does not agree, and the decision to market is BRL*s. None- 
theless, there is some uneasiness about the relationship of Banneker^s 
program to BRL*s marketing efforts. The negotiation of a royalty ar- 
rangement Improved but did not completely eliminate this uneasiness. 

THE EFFECT OF THE STRIKE 

The 22-day strike in the late spring of 1972 was apparently unre- 
lated to the American Federation of Teachers' disapproval of performance 
contracting but Instead was a response to the financial crisis of the 
Gary school system and the school board* s proposals to deal with it. 
Even though some of the lost days were made up, the school year was not 
long enough to satisfy the provision of the 150-day school year that the 
School Clty-BRL contract requires for the achievement "guarantees." 

The first issue that arose concerned post-testing. Gary had a 
teacher strike in 1970 that had serious adverse impacts on achievement 
test scores. BRL argued that to post-test at the end of the school year 
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soon after the strike would be meaningless and e^'en misleading. After 
discussion between School City and BRL, It was decided to cancel the 
scheduled post- test for payment and evaluation purposes. The Banneker 
students were, however, administered the same district-wide tests that 
all other Gary students received In the spring of 1972. Longitudinal 
comparisons of Banneker students with 1970-71 results on computation of 
pre- and post-achlevcment gains for 1971-72 will not be possible. 

The second Issi e concertied how to arrange payments for the students 
who were graduating from the sixth grade in June 1972, as well as pay- 
ments for those In the program for all three years. This question be- 
came Intertwined with the Issue of how to deal with the declining en- 
rollment. Banneker only had 707 students In 1971-72, and It appears 
unlikely that In 1972-73 the contract requirement for 700 students will 
be achieved. At the time this report was prepared, the contract adjust- 
ment details had not been announced. 

BANNEKER SCHOOL AND EDUCATIONAL INNOVATION 

In exploring the Issue of the relationship between performance 
contracting and educational change, we formed a somewhat complex Im- 
pression. There Is no likelihood that Banneker will be a model for 
other Gary schools. The controversy over the program and Its results 
ensures against any simple replication In other schools. Banneker- 
developed materials and curricula might see some use In other Gary 
schools, but even here the diffusion will probably be limited. Gary 
principals share a "not Invented here" syndrome and do not like to 
adopt another school's program without extensive modifications. More- 
over, In Gary the cleavage between the black and white communities has 
led the white schools to regard programs developed In black schools as 
Inappropriate to their educational needs. Consequently, although some 
components of the Banneker system may be adopted In other Gary schools. 
It Is hard to see that there will be any widespread tumkeylng. 

Even so, Gary administrators argue with conviction that the pro- 
gram has led to some system-wlde changes. One argument Is that the ap- 
proval of the BRL contract was a signal from the school board that In- 
novation was hl^ly regarded and would be supported by the administration. 
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Accordingly, school personnel sponsored a number of innovative programs c 
system changes and generally became more responsive to new ideas. Exam- 
ples include a new program of in-service training during released time. 

In an almost Hegelian dialectic fashion, the same people who argue 
that the Banneker program has been a force for change also point out 
that it may have made further change more difficult. First, as a re- 
sult of the demands attendant on Implementation of Banneker 's program 
and the other new programs, the Gary school board has grown more inter- 
ested in perfecting current programs and less interested in bold new 
ventures. Perhaps even more important, the Banneker program sensitized 
the Gary Teachers Union and the Indiana State Department of Instruction 
to the possibility of their interests being affected by new programs. 
Both are monitoring School City activities more closely. As a result 
of the Banneker experience, such groups are also likely to be more ef- 
fective in any future disputes. Finally, the State Department of In- 
struction is sponsoring legislation that would give it more authority 
in any future dispute such as that over the BRL contract. In short, in 
the near future any program as atypical as the Banneker Curriculuii Cen- 
ter may well have a harder time getting by the school board, union, and 
state education officials. 

REACTIONS TO THE 1971-72 PROGRAM 

Some general reactions of informed observers have already been 
mentioned. The reactions of two individuals have special interest and 
will be summarized here. 

Sandra Irons, President of the Gary Teachers Union, stated that 
at the start of the program the union opposed Banneker on two grounds. 
The first was a general opposition to performance contracting, and the 
second was opposition to the way the program was implemented in Gary. 
After the program was accepted over its objections, the union concen- 
trated on the question of the way it was implemented. During 1971-72 
the union was involved in two disputes: The first was over the proposed 
reduction in staff that would have violated ser.lority arrangements: 
the second was o^er the validity of the 1970-71 evaluation report. The 
union prepared a report analyzing the evaluation report of the Center for 
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Urban Redevelopment in Education (CURE) . The report criticized the lack 
of comparison or control groups. It also questioned why math gains were 
higher than reading gains. 

On the basis of the union's analysis. Irons feels that the students 
who went Into the program near or above the national norms did better 
than the ones who were far below these norms . She said that In the 
summer of 1971 several teachers worked on BRL's math programs. One 
such teacher examined the evaluation test to check whether materials 
covering the subjects being tested were included in the program. Ap- 
parently compound fractions, for example, were not Included in the in- 
structional material even thougji they were Included on the test. It 
should be noted that before taking the fMll-tlme Job as union president. 
Irons was a math teacher, and this area of the Banneker program is of 
particular Interest to her. 

The union still regards the program as differentiated staffing. 
Irons said there was supposed to be no merit pay, but it turns out, ac- 
cording to her, that the curriculum managers have more authority and 
freedom than the teachers. Some of the teachers complained that when 
class sizes were large, the curriculum managers did not do their share 
of the teaching. Some of the teachers felt that the aides were trying 
to control the classrooms, and conversely some of the aides felt that 
some of the teachers were trying to downgrade them. The Indiana State 
Department of Instruction has been talking about certifying the aides 
and requiring sixty hours of college instruction. Irons does not like 
this, but thinks that next year the State Department will specify guide- 
lines for aides that will tend to upgrade the qualifications required. 

Irons was asked about the drive to unionize aides. She stated that 
the union had more pressing problems during 1970-71 and had not been 
able to devote time to this issue. 

Irons' summary opinion of the Banneker program is that it is just 
another innovation without much merit. She says that Banneker teachers 
are doing the sane things that teachers elsewhere are doing without per- 
formance contracting. As long as school district* ' ave the kind of finan- 
cial problems that Gary has had, she feels they will have programs such as 
performance contracting, which are sold to the districts as money -s avers . 
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The Banneker principal, Sherman Newell, saw BRL acting as a catalyst 
for change. He said that unfortunately the only times that changes occur 
in education are when they are brought in from outside. We asked him to 
distinguish between what he saw as positive changes and what he saw as 
negative changes or problem areas. Starting with the positive, he listed 
differentiated staffing, which Includes the Increased use of paraprofes- 
sionals in the classroom and a sophisticated departmentalized system of 
instruction that allows teachers to become specialists. He attributed 
to the Banneker program the introduction of the Allyn and Bacon materials 
Into the social studies curriculum. He was Impressed that the way Ban- 
neker is organized allows the principal to be the real Instructional 
leader and fulfill the role of providing assistance to the teacher. 

As to the problem areas, Newell stated that bringing in an outside 
agency is an affront to the local professional staff, with the implica- 
tion that the administration thinks that the staff Is not doing a good 
Job. Most of the other problems he listed were in the area of "people 
problems." He noted that performance contracting I3 cognitive-oriented 
and thus there must be concern about ensuring attention to the affective 
areas. He said that when people are entrenched in traditionalism. It is 
hard for them to change: "If you do not really change them, what you do 
in an experience like Banneker is to show them some new strategies for 
coping with their professional problems." 

S UMMARY 

The teachers' strike in the spring of 1972 meant that the payments 
and evaluation plans for the three-year performance contract had to be 
revised. How this revision will be made and how It will affect the na- 
ture of the program had not been publicly released at the time this 
report was prepared. 

The Gary performance contracting program has had three unusual 
features: First, BRL has had cognizance over the entire Banneker school, 
permitting more flexibility in scheduling, organization, and curriculum 
than Is typical in other performance contracting programs. Second, the 
contract between BRL and School City provides for a three-year program 
permitting more time for program development than has be«n usual elsewhere. 



-41- 



Third, the program objective Is to have all Eanneker students performing 
at grade level in reading and mathematics; BRL is to receive no payment 
for students who after three years do not test at the national norm for 
their grade. 

In 1970-71, BRL made a number of changes in the Eanneker organiza- 
tion and governance. In 1971-72 BRL's stress was on curriculum develop- 
ment, and the organization and governance were more conventional. BRL's 
emphasis has been on development of improved curricula for language 
arts and mathematics. 

The advantage of having a multiyear program, as well as the applica- 
bility of the achievement "guarantees," were called into question by the 
system-wide 22-day strike of the Gary Teachers Union. The loss of time 
meant that School City could not meet the contract requirement to provide 
BRL with 150 days of instruction time. The strike also meant that it will 
be difficult to compute the "success" of the program. Even before the 
strike, we were told by several knowledgeable sources that the goal of 
all students testing at grade level was Utopian. More "realistic" fore- 
casts, such as one-half or one-third of all students attaining grade level 
reading norms, were being mentioned both before and after the strike. 

Unless the test results from the program are spectacular, it is un- 
likely that Banneker's program will be replicated in other Gary schools. 
Despite statements in the contract, informed observers expect that half 
or less of the Banneker students will qualify for performance payments, 
and it seems unlikely that such a result would be regarded as "spectacu- 
lar." Therefore, it is unlikely that there will be much diffusion of the 
BRL program as a system. However, features of the system have been and 
probably will continue to serve as models for changes in Gary school 
procedures . 

Gary officials believe that the program has been a force for change. 
The essence of their belief is that the program announced to Gary school 
teachers and administrators that the "name of the game" wab innovation 
and indeed there was an appropriate response. However, another con- 
sideration was voiced, even by some of the officials who had used the 
above argument. The attention and publicity given the Banneker program 
may have made future innovations more difficult to implement. The 
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thinking here is two-part: First, the Gary School Board may be less will- 
ing to engage in another, bold venture in the near future; second, the 
State Department of Education and the Teachers Union have become alerted, 
sensitized, and experienced in dealing with Gary innovations. They will, 
the reasoning goes, be more likely to monitor Gary's actions and block 
new programs of which they disapprove. 
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V. GILROY. CALIFORNIA 



INTRODUCTION 

Gllroy Unified School District (GUSD) had a performance contract 
with Westlnghouse Learning Corporation (WLC) In 1970-71 for 103 students 
In the second, third, and fourth grades for both reading and math. The 
program was discontinued at the end of the 1970-71 academic year. There 
had been serious Implementation problems, but many of the operating prob- 
lems that had been encountered had been smoothed out, and a cooperative 
effort between the school district personnel and the WLC on-site direc- 
tor was resulting In program Improvement. Enthusiasm of the staff at 
Eliot School, which housed the performance contract, was high at the end 
of 1970-71. The Eliot staff wanted to carry out Its own resource center 
during 1971-72; however, the program was not funded because Eliot went 
on double session and no discretionary funds were available. 

Despite termination of the WLC program, Gllroy had a performance 
contracting program during 1971-72. The principal of Rucker School 
adopted the Behavioral Research Laboratories' Sullivan Reading Materials 
and selected BRL's performance option for payment purposes. 

REACTIONS TO THE 1970-71 PROGRAM 

The Eliot teachers, the principal, the director of elementary In- 
struction who had primary cognizance of the program, the director of 
special projects, and the GUSD superintendent agree that although achieve- 
ment gains had fallen far below their expectations, the performance con- 
tracting experience had a salutary effe.t on the district. They assert 
that teachers are making more us« of Individualized Instruction than 
they did In the past; teachers and principals are more receptive to try- 
ing Innovations in teaching and In classroom organization, and teachers 
are more willing to have the results of their classroom efforts judged 
by the achievement gain of their students. 

Teacher 1 t The only negative reaction we encountered to the per- 
formance contract was from one of the two teachers Involved In the 
1970-71 program. She Indicated that she Is now very wary of programmed 



instruction because she believes that the material is not suitable to the 
children she teaches. She felt that the solution to Gilroy's reading 
problem is to rely heavily on teacher-made materials when working with 
youngsters whose vocabularies are severely deficient. 

Teacher 2: The second teacher felt that the performance contract 
had forced Eliot teachers to individualization of instruction. She 
said that the performance contract made the children more independent 
in the way in which they operated in the classroom, and that in 1971-72 
most of the teachers in the school had contracts of some form or another 
with the children. They were also making heavy use of the Distar pro- 
gram, which they feel is good for bilingual students because there is 
constant teacher-child contact, and they believe it teaches children 
how to listen better. 

Generally, teachers thought that Westinghouse had been thrust on 
them in 1970-71, although toward the end of the year most of the ill 
feelings had disappeared. The second teacher mentioned that Westinghouse 
should have had an educator and not a businessman as program manager. 
She thinks that the improvement in classroom instruction at the school 
in 1971-72 was largely a result of the Westinghouse experience. Teachers 
are, according to her, more child-centered and are providing much more 
individualization of instruction. They also made extensive use of games 
in both math and reading at Eliot School in 1971-72. 

The teacher's final comment was that while the first- and second- 
grade results are often disappointing, what was accomplished in those 
two grades is often reflected in the third gr^^de. As will be discussed 
later, this observation was supported by the results of the achievement 
testing. at Eliot in the third grade this year. 

The Principal of Eliot School: Robert Medley felt that there had 
been many positive spin-offs to his school and faculty because of the 
performance contract last year. Primarily, there is more indiviJuali- 
zation of instruction. The faculty has developed and implemented a 
diagnostic-prescriptive system in order to test, diagnose difficulties 
and strengths, and prescribe an instructional strategy for each student 
in the school. There is more openness towai ^ new ideas on the part of 
the faculty. He noted that most of the teachers have been using some 
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fonn of Individual contract with their students In both reading and math. 
In addition, certain teachers have been using some kind of tangible re- 
ward system (a feature of the WLC program). Including items such as toys 
and candy, and other teachers have been using point systems. 

finally. Medley felt that he got many Insights Into the problems 
of how to measure teacher success In terms of achievement results. He 
feels that this will be useful to him In UgU of California's Stull 
Bill, which requires all school districts to Implement a system of 
accountability that will measure teacher success In relation to the 
educational achievement of their students. 

The Director of Elementary Instruction: Rodney T. Kelley also 
felt that there had been a good spin-off to the district from the WJ.C 
experience. He went to Eliot School with the results of 1970-71 's 
achievement testing and asked what services the central office staff 
could provide to help teachers solve their problems. As an outgrowth 
of that conversation, the faculty decided to Implement school-wide an 
Individual, diagnostic-prescriptive system for all students. Although 
teachers leaned heavily on what they had learned from WLC, they also 
developed many of their own materials and bought others that they con- 
sidered to be better than what hac* been used the previ .s year. Kelley 
Indicated that the faculty may have wanted co show that it could Indeed 
do better than WLC had done, and he felt that this was another positive 
spin-off. 

Both Medley and Kelley discussed this year's achievement testing 
In glowing terms. They said that a substantial proportion of the stu- 
dents in the Title I program at ..Hot School had made at least a month's 
ga'-i for a month's instruction. While they both felt that it is diffi- 
cult to definitely attribute change to any single cause because many 
Innovations are being carried out simultaneously in the district, they 
did give Westlnghouse a great deal of credit for having acted as a cata- 
lyst for change in the district. 

The Director of Special Projects: According to Dave Downing, the 
WLC experience made teachers in the district more .-ware of, and more 
concerned about, academic achievement and the need to find ways of im- 
proving it. He feels that teachers are now more willing to be measured 
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by the achievement of their students, and are beginning to shed the 
notion that educational outcome cannot be judged by test scores. This 
has led to district-wide pre- and post-testing in every grade from kin- 
dergarten to the sixth. He said that in a school such as Eliot, 15 or 
20 children were ordinarily recommended in each previous year for spe- 
cial classes because the teachers felt they were too difficult to han- 
dle in the regular classroom. The approach the teachers are now taking 
toward difficult children is to discuss the nature of the problem with 
the psychologist, and to seek his help in diagnosing the child's particu- 
lar needs in order to find ways of helping him in the regular classroom. 

Downing is still skeptical about the involvement of private enter- 
prise In education. On the other hand, had the results at Eliot Deen 
different in 1970-71, he would have instituted as many performance con- 
tracts as seemed appropriate in the light of the district's needs. 

rha Superintendent of GUSDx Robert Infelise felt that performance 
contracting was one of the reasons that GUSD is finding an Increasingly 
positive reaction on the part of teachers to the idea of accountability 
in the district. Teachers do not feel as threatened by the idea as in 
the past. Furthermore, they seem comforted by the fact that an outsider 
was not able to get better results than they have obtained. They are 
willing to face and to cope with the fact that there has not been great 
success in dealing with their population of students. 

Infelise said that in dealing with Westinghouse the school found 
that there was difficulty in making the changes that all participants, 
Including the program manager, felt would be beneficial to the pro- 
gram. Infelise ventured the opinion that WLC's structure locked the 
concern into a framework that sometimes prevented implementaMon of 
logical solutions to problems because WLC felt that it was necessary 
to "stick to the agreement." For example, early in the 1970-71 pro- 
gram, both the on-site manager and the program teachers wanted to 
change the class sizes. There was a long time lag before the switch 
was made because the decision had to be made at a central level rather 
than on site. 
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Infellse pointed oat that Gllroy had entered Into another contract, 
this time with the Sullivan Reading Centers Division of BRL. The prin- 
cipal of Rucker School negotiated the contract although It was signed 
by the superintendent. When asked how this performance contract came 
about, Infellse said that school officials are trying decentralization 
of authority In the district, with participative management by the prin- 
cipals. Budgets have also been decentralized so that each building prin- 
cipal has some autonomy in the way he spends his share of the funds. 

Infellse also stated he felt that performance contracting was 
greatly influential in bringing about more individualized instruction 
throughout Gllroy. 

THE 1971-72 PERFORMANCE CONTRACT 

In January 1972, Mary Flautta, the principal of Rucker School, ne- 
gotiated a contract with the Sullivan Reading Center of BRL to teach 
reading to thirty students in the Title I program. The 12 third-, 6 
fourth-, and 12 fifth-graders who participated in the program were 
chosen by the teachers and principal as being the most in need of re- 
medial reading help. 

When asked why she had decided on a performance contract, and why 
she had chosen the Sullivan Reading Center, Flautta said that Sullivan 
materials had already been used in the school and proved successful. 
She also liked the fact that four of her teachers could observe the 
program and be able to help Implement Improvements on their own. 

Under the guarantee, Sullivan supplied a teacher who worked with 
two groups of fifteen students each for an hour a day. Paymer as to 
be made only for a student who has gained nine months, and the guarantee 
was voided for any student who did not attend at least 5/6 of the appli- 
cable portion of the school year. 

The basic technique used by the Sullivan toicher was to work with 
the entire group of fifteen children for abvjt ten minutes on a problem 
common to all the children. For the more advanced children, this was a 
period of drill, and for the less advanced, a period of new learning. 
Then each child worked on his own, and the teacher circulated to see 
what each child was doing, corrected mistakes, and supplied new materials 
to those who needed them. 
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When asked what the teacbers who observed the Instruction thought of 
the system, Flautta said that they had mixed reactions. They could see 
the possibilities, but felt it would be extremely difficult to implement 
ill a classroom of more than fifteen children. 

Only two of the thirty children made the guaranteed gain of nine 
months. The average gain for the twenty-nine students (one moved) was 
one month. Of the students in the program, fourteen gained, twelve 
lost, and three showed neither a gain nor a loss.* The teachers felt 
that the benefit of the program to the children was greater than that 
reflected by the gain scores, both in terms of learning and in the de- 
velopment of self-confidence. 



ACHIEVEMENT AT ELIOT SCHOOL IN 1971-72 

The results in 1970-71 were less than expected, but they were com- 
parable with the normal Title I gains in Gilroy, and in some cases bet- 
ter. What was the achievement of the students at Eliot in 1971-72 
without the performance contract? We obtained the results of the testing 
for 1971-72 for students in the third and fourth grades, but because 
different tests were used this year from last, it is meaningless to re- 
pcrt actual scores. We did, however, compute reading and math gain scores. 

For analysis of the 1971-72 results the students were divided into 
those who had participated in the WLC program last year, and those who 
had not. In reading, second-grade students in the WLC program gained 
0.37 in 1970-71; in 1971-72 in the third grade these students gained 1.1 
years. Nonprogram students in the second grade in 1970-71 gained 0.7; 
in 1971-72 the third grade exclusive of the WLC students gained 1.3 years. 
It should be borne in mind that the children originally chosen for the 
program were those most in need of remedial help.^ Third-grade students 

There are serious questions about the appropriateness of using a 
standardized achievement test to measure the effect of so short a period 
of instruction, and moreover to use such a teat for a purpose other than 
that for which it was designed. For discussions of these problems see 
P. Carpenter and G. R. Hall, Case Studies in Eduao.tional Performance Con-^ 
tracting: Concluaiona and Implications^ The Rand Corporation, R-900/1-HEW, 
December 1971; and S. A. Haggart^ G. C. Sumner, and J. R. Harsh, A Guide 
to Educational Performance Contracting: Technical Appendix^ The Rand 
Corporation, R-955/2-HEW, March 1972* 

^ ^This point is emphasized by GUSD officials. 

ERIC 
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In WLC m 1970-71 gained 0.6; in 1971-72 in the fourth grade these students 
gained 0.76. Nonprogram students in the third grade in 1970-71 gained 
0:2; in 1971-72 the fourth grade exclusive of the WLC students gained 0.6. 

Note that the rate of gain for 1970-71's second-grade program stu- 
dents tripled the next year; for 1971-72 's nonprogram students it al- 
most doubled the next year. The students who had been third-grade pro- 
gram students in 1970-71 did only slightly better in the fourth grade 
in 1971-72, while the 1970-71 nonprogram students tripled their gains. 

In math, the same pattern is observed. Second-grade students in 
the WLC program in 1970-71 gained 0.45; in 1971-72 in the third grade 
these students gained 1.2 years. Nonprogram students in the second 
grade in 1970-71 gained 0.7; in 1971-72 the third grade exclusive of 
WLC students gained 1.4 years. Third-grade students in WLC in 1970-71 
gained 0.67; in 1971-72 in the fourth grade they gained 0.65. Nonprogram 
students in the fourth grade in 1970-71 gained 0.7; in 1971-72 the fourth 
grade exclusive of the WLC students gained 0.82. 

In both reading and math, two interesting phenomena were observed. 
Firsts both former WLC students and students who had not participated 
in the program did extraordinarily well in third-grade reading in 1971-72; 
and second, none of the groups did better than expected in the fourth 
grade during the same period, with the exception of the group who had 
not participated in the program in the previous year who tripled their 
reading gains. 

We cannot attribute changes during 1971-72 to specific events with 
any certainty. We can only speculate that the teachers in Eliot School 
did, indeed, benefit from the performance contracting experience. We 
were told by the teachers and by others that they were individualizing 
instruction, relying on a diagnostic-prescriptive approach, and trying 
hard to improve Instruction. The results, especially in the third 
grade, could be interpreted as evidence that their efforts were success- 
ful. Why this was not true in the fourth grade is far from evident. 
In the past we have encountered similar situations in other school dis- 
tricts. Educators have speculated with us about the cause of "fourth- 
grade slump'* and have offered two possible explanations: First, in the 
fourth grade most achievement tests switch from having students mark 
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thelr answers In the test booklet to using machlne-^s cored separate answer 
sheets; second, many teachers clamp down hard in the fourth grade on 
the Idea that school Is for real, that the children are no longer to 
make a game of their work, and that education Is a serious business. 
It is difficult for many youngsters to adapt to the new classroom situa- 
tion and atmosphere. The hypothesis needs to be tested. 

SUMMARY 

Having a performance contract in Gllroy last year appears to have 
served as a catalyst for change in the school district. Teachers are 
implementing diagnostic-prescriptive procedures as a basis for individ- 
ualized instruction; they are more receptive to trying innovations, and 
they are more willing to have the results of their classroom efforts 
Judged by the achievements of their students. 
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VI. GRAND RAPIDS, MICHIGAN 



THE 1971-72 CONTRACT LEARNING PROGRAMS 
Introduction 

The three performance contractors of 1970-71 were retained In 
Grand Rapids for 1971-72 with seven contracts covering learning centers 
In fourteen schools. Only three of the contracts (three schools) had 
guaranteed performance provisions. Contract provisions between the 
contractors and the Individual schools are detailed below: 

Table 5 

CONTRACT LEARNING PROGRAMS, 
GRAND RAPIDS, MICHIGAN: 1971-72 



Contractor 

CMES 

Learning Unlimited^ 

Alpha 11^ 

Alpha II 
Alpha II 

Alpha II 
Alpha II 



Schools 

South Middle 
Colt Elementary 
Franklin Elementary 
Lexington Elementary 
Sibley Elementary 
Straight Elementary 
Alexander Elementary 
Fountain Elementary 
Hall Elementary 
Kensington Elementary 
West Middle 
Burton Junior Hlgli 

Alternative Education 
Center (social 
rehabilitation) 

Coldbrook School 
(mentally 
handicapped) 



Type of 
Contract 

Performance 
Fixed fee 



Consultation 
only 



Performance 
Consultation 

only 
Fixed fee 



Performance 



Maximum 
Contractor 

Fee ($) 

120,000 
150,000 



23,000 

52,000 
6,000 

22,000 
84,000 



ERLC 



1Q-,^^f "^'^^f^^^^ organized as a corporation in the spring of 

1971 when Westinghouse divested Itself of its contracted instruction- 
apparently all of the Learning Unlimited personnel, at least at the op- 
erating level, came over from Westinghouse. 

Alpha, the contractor for 1970-71, reincorporated for tax reasons 
and was renamed Alpha II. 
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The programs at Kensington Elementary and Burton Junior High were 
for reading only, the program at Alexander used only the token economy, 
and the other programs were for both reading and math. 

The programs in these fourteen schools represent a wide range of 
options for the evolution of a performance contracted program. The 
programs at South Middle and West Middle were essentially replications 
of previous performance contracting programs. The programs at Franklin 
and Lexington schools were replications of the first year, but with 
fixed fee rather than performance payments, A^Sibley and Straight, the 
programs were replications of previous fixed-fee programs. Hall School 
had a turnkey program with some modifications in physical arrangements. 
Alexander School turnkey ed its 1970-71 program but retained very few 
elements of the original program, Coit, Fountain, Kensington, and Bur- 
ton schools adapted programs that had been used in other schools with 
varying degrees of modification. Alternative Education (fixed fee) and 
Coldbrook School (performance payment) both represent novel applications 
of the program of a previous contractor. This diversity reflects the vary- 
ing perceptions of needs in the different schools. It is also indicative 
of a district policy to encourage initiative at the building staff level, 

CMES; South Middle School 

The performance payment provisions were retained for a second year 
and there were very few changes in instructional format. The major 
change was in student selection. In 1970-71, participants were se- 
lected from the lower achievers of all four grades (six through nine) , 
Last year, all sixth-graders and selected seventh-graders received read- 
ing and math instruction in the learning centers; selected eighth-graders 
received math instruction only. 

The program will be turnkey ed in 1972-73, since CMES is going out of 
the performance contracting business. The district will buy all the 
equipment and software, and will negotiate for CMES-proprietary items 
such as diagnostic materials, prescriptive materials, and reading manuals. 

The grade gains in 1971-72 for sixth-graders, who as a group were 
almost two years behind grade level at the first of the year, were only 
0,3 in reading and 0,1A in math. Seventh-graders, who were over three 
years behind grade level, gained 1.7 In reading and 0.9 in math. 
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Leamlng Unlimited learning centers were operated in five public 
schools and one parochial school (St. Andrews Elementary) that was at- 
tached to the Grand Rapids School District contract to enable the paro- 
chial school to take advantage of the group rate offered by the con- 
tractor. The organization and operation of the learning centers differed 
only slightly from the original year. At Coit School, the regular 
teachers instructed their respective students in the learning center. 
At each of the other schools, a staff member was assigned exclusively 
to provide instruction in the center. Elmer Vruggink, Assistant Super- 
intendent for Instruction, regards the latter arrangement as more "cost 
effective" but thinks that the nonparticipation of the regular teachers 
made them feel less a part of the program. 

The contract specified achievement and interim mastery testing as 
modes of evaluation. The stated goals were that a full grade equivalent 
(reading or math) would be gained by 90 percent of the students who at- 
tended at least 160 sessions (reading or math), and that the average 
gain for each year of attendance would be 1.5 grade equivalents. Con- 
tract payment was not dependent on evaluation results. 

Learning Unlimited offered volume rates. The 1971-72 schedule was: 

First center $34,500 

Second center $20,500 

Third center $18^500 

Each additional center $17,500 

For the six learning centers (including St. Andrews), the average 
price was $21,000. This price included the instructional and motiva- 
tion system, the diagnostic and prescriptive services, teacher training 
(pre-service and in-service), and ongoing consultation and administrative 
services. An additional $5000 per center was charged for educational 
and motivational materials. The fee was paid in ten equal monthly in- 
stallments beginning August 31, 1971. 

For 1972-73, one additional elementary school was added to the list 
of schools with Learning Unlimited learning centers. The new price 
schedule offered was substantially lower than that for 1971-72, ranging 
from $15,500 for the first school to $6000 for the eleventh school. 
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The additional cost-per-school for materials dropped somewhat to $4500. 
The average total fee was $14,000 per school and the services were ap- 
parently the same as In 1971-72. 

Grade gain achievement was mixed for the several Learning Unlimited 
schools. With a few exceptions gains over the previous year (there was 
no fall pre-test) were generally in the neighborhood of 1.0. The fourth- 
graders at Lexington Elementary gained in the neighborhood of two grade 
levels (2.0) in reading and also in math. Lexington also scored a con- 
spicuous success on the State of Michigan educational assessment, a 
state-wide testing program for fourth- and seventh-graders. The Grand 
Rapida Press (July 14, 1972) reported that Lexington's city-wide rating 
rose from 39th in 1970 to a tie for first place in 1972. The principal 
at Lexington attributed much of the success to the Learning Unlimited 
learning center. 

Alpha II; Elementary 

Alpha conducted a two-week workshop at the end of August 1971 to 
train teachers from schools that were planning Alpha learning centers. 
The fee for the sixty-hour workshop was set at $144 per participant or 
$17,640, whichever was greater. Price reductions were offered if par- 
ticipation exceeded 160 persons. 

The workshop was planned primarily for teachers from West Middle 
School, Burton Junior High, and the Alternative Education Center; these 
programs will be discussed later. In addition, teachers from four ele- 
mentary schools (Alexander, Fountain, Hall, and Kensington) were invited 
to attend by the district. The idea was to familiarize the teachers 
with the Alpha system and let them decide for themselves whether and 
how the program could be implemented in their respective schools. They 
were told they could adopt as much or as little as they pleased, and 
that they could proceed with the implementation at their own pace. 

The teachers from Fountain and Hall chose to implement the complete 
Alpha program as it was in the OEO experiment, except in individual 
classrooms rather than in learning centers. This made the program some- 
what more costly than before, because materials were split more ways and 
it was necessary to provide ongoing consultation to twelve teachers 
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rather than to four. Kensington adopted the complete program, but only 
for reading; Alexander opted for the token economy only, but tokens were 
used for all instruction, not just for reading and math. 

Except for Alexander School, all contracted programs replaced 
regular programs. Remedial reading teachers were reassigned to work 
in the Alpha program. 

According to Vruggink, the teachers, despite the ground rules, 
expected the program to be laid out for them. Some of the materials 
did not arrive until November because they were not ordered until the 
teachers had decided what to do. The situation was somewhat aggravated 
by the turnover of Alpha consultants. By spring, however, the programs 
were moving along, and all four schools wanted to continue the programs 
in 1972-73. 

On-site Alpha personnel felt that the programs in the four ele- 
mentary schools compared well with the preceding year, considering that 
sixty teachers were supported by one consultant, whereas in 1970-71 
there were only eight teachers supported by an on-site manager. The 
teachers had to assume much more responsibility than anticipated and 
compromised the system somewhat. For example, Alpha reports that teachers 
did not use diagnostic elements as recommended, and sometimes the "rule 
book" was regarded too literally and students were moved ahead at slower 
rates than necessary. Some elements were simply not used, especially 
those that were late in arriving. Coordination became a big problem 
without the on-site manager, even though teachers had definite respon- 
sibilities. One Alpha employee speculated that, as of the end of the 
school year, 70 percent of the teachers were enthusiastic over the pro- 
gram, but the rest were still concerned because the system was new and 
there was too little consultation; the employee felt that teachers in 
the latter group had weaker innate management skills. There were a few 
instances where the teachers improved on the Alpha program, such as 
* where quiet reinforcement activities were placed in the same room as 
instruction. 

The fee for consultant support for the four schools was $22,791. 
This was paid in four equal installments in September, Novenfcer, January, 
and June. Grade gain data for the four schools has not been made public. 
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Alpha II; West Middle School 

The Alpha program at West Middle School essentially replicated the 
program of the previous year, except that the number of student units 
(one student enrolled In one subject) Increased from about 600 to 800, 
and the program was not strictly remedial. About 240 seventh-graders 
were enrolled for reading only, and enough other students were enrolled 
for reading and/or math to coiiq>lete the remaining 560 student units. 
When available, reading scores from the previous year's achievement 
testing were substituted for pre-tests. Grade gain in reading for the 
seventh-graders was less than half a year. Average grade gain in math 
for all three grades was somewhat less than a full year. 

The payment provisions were complicated. Briefly, there were 
separate performance-based payment schedules for seventh-grade reading 
students and for "serious academic retardates" with good attendance 
records. The fee for the remaining 300 or so student units was fixed 
at $17,050. The maximum payment for all students was set at $52,000. 
There were threji interim payments of $12,000 each (September, December, 
and March) . 

West Middle staff members have pressed to turnkey the program for 
1972-73. They want the flexibility to institute modifications, such 
as allowing students easy entry to and exit from the program. Some of 
the concerns in the turnkey decision are increased remuneration for the 
person assuming leadership, additional supplies for more program flexi- 
bility, and relocation of the Alpha II office to nonpublic property. 
The school would continue to purchase student flow-charts from Alpha, 
and presumably would contract for consulting services in the same man- 
ner as thi'. elementary schools. 

Alpha II: Burton Junior High 

Alpha established a reading skills program at Burton Junior High for 
400 seventh- and eighth-graders, involving three teachers and two aides. 
Alpha provided all materials and student incentive funds for $5825. 
The comment was heard from several quarters that Burton had achieved 
the "model" program in terms of student achievement and behavior. One 
factor may be the attendance area. Burton's area is probably more 
^ middle class than the areas of the other schools with learning centers. 

ERIC 
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Average grade gain over the previous spring was about one year* The 
principal at Burton reported that the majority of the staff felt that the 
program could be turnkeyed and operated internally. The staff would like 
to use any savings from a turnkey operation to expand the program. 

Alpha II; Alternative Education 

Russell Hormellnk Is Project Director for Grand Rapids' Alternative 
Education Center. The Center Is for male students who are behavior 
problems In the regular schools and who are wards of the juvenile court. 
The usual mix of school instruction is provided for 130 students, grades 
six through twelve. Entrance Is voluntary (transfers from regular 
schools or dropouts over the age of sixtec^n) ot involuntary (indefinite 
suspensions from regular schools and juveniles who are ordered to attend 
by a probate judge). A student is removed from the program if (1) he 
completes a contract that indicates he can be successful in a regular 
school, (2) he is too much of a behavior problem, or (3) his attendance 
rate is below 50 percent and he is more tha.i sixteen years of age. The 
enrollment is limited, and there was a waiting list of forty students 
at the time of our visit. 

For the 1971-72 school year. Alpha II contracted to provide instruc- 
tion in reading and math with a program similar to that at West Middle 
School. A fixed fee of $5210 was charged to equip a reading room, a 
math room, and a free room, together large enougih to accoinnodate 150 
students. An additional $5 per enrolled student was charged at the end 
of each of 6 six-week periods. Alpha II paid for classroom materials, 
free-room materials, student rewards, and visiting Alpha staff, and 
on-site support was provided by the resident Alpha consultant at West 
Middle School. Grand Rapids provided classrooms, normal school services, 
at*<? salaries for its staff. The Alternative Education Center staff 
memt3rs attended the Alpha II Summer Workshop during the last two weeks 
of August 1971. At the end of the school year, 115 of the 127 students 
in Alternative Education were attending Alpha's learning center. Tlie 
total contract liability of the board of education for both the learn- 
ing center and a social rehabilitation program was not to exceed $21,900, 
equivalent to the combined fees for a full enrollment of 150 students 
during each of the 6 six-^eek periods. 
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Robert Stark, who is the district's Director of Pitpil Services and 
directly responsible for Alternative Education, was very positive about 
the Alpha II learning center program. He reported that contractual per- 
formance guarantees were ruled out because the district did not want 
the learning center to interfere with the flow of students in and out 
of Alternative Education, which usually takes place at the end of the 
six*week intervals* 



Alpha II: Coldbrook School 

Grand Rapids had a novel performance contract with Alpha II for 
reading and math instruction at Coldbrook School, a special school for 
the educable mentally handicapped (EMH) . The district has three-year 
funding from Title III to "devise and implement a better educational 
model" for this category of student. The program is intended to pro- 
vide a more adequate emotional and social adjustment for the students, 
to reduce the per capita cost of instruction, and to increase scholastic 
achievement. By the end of the three years, the program will be decen- 
tralized, and the approximately 150 students at Ccldbrook will be ac- 
commodated in regular school buildings and in regular classrooms. The 
regular schools will have learning centers, but these will not be ex- 
clusively for EMH youngsters, as it is hoped the "special education" 
stigma will disappear. 

The first year of the project was 1971-72, and the main objective 
was to develop better teaching methods in reading and math. Alpha op- 
erated learning centers for both reading and math. Each center was 
staffed by one teacher and two aides. Each student spent a half-hour 
per day in each center, plus a half-hour on each of the two subjects in 
regular classrooms. Tokens were used to reinforce good school perfor- 
mance and behavior in the learning centers and regular classrooms for 
work on reading and math* The tokens were redeemable at the "Alpha 
Store," which was open once a week, or were saved up for special recre- 
ational activities outside the school. Robert whltecraft, the district*s 
project director at Coldbrook, said that token rewards had been discon- 
tinued as behavior inducement after four or five months, when the need 
for this type of reinforcement diminished and the tokens became strictly 
a reward for academic performance. 
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There were 145 students at Coldbrook, from five to sixteen years of 
age. The average IQ was 59, and average grade gain had been only one 
or two months per year. Children were grouped into ten classrooms ac- 
cording to age, with a maximum of fifteen per r>om. The spread in a 
classroom was as much as seven grades, so there was a necessary emphasis 
on individualized instruction. 

The contracted performance goal for each student was 1.5 years* 
achievement gain in each subject. For each such goal attained. Alpha 
was to receive $231.04. Payment would be $12.16 more or less thar 
that amount for each 0.1 gain above or below 1.5 years. The maximum for 
any one student per subject was set at $29.1.84. The maximum payment 
(figured for 125 students) for achievement gain in reading and math was 
not to exceed $60,800. Alpha was also to be paid up to $7600 for each 
of the two subjects, depending on how students performed on interim 
performance ob^-ictive testing; the contract was obscure on the formula 
for determining the exact amount. In any case, the maximum total for 
achievement gains and performance objective ga-t.ns was $76,000. There 
were four interim payments of $10,000 each. 

In addition to these fees. Alpha was provided $7600 before che 
start of the school year as "risk capital" for the development or modi- 
fication of its program for the EMH students. Half of this amount was 
to be refunded at the end of the school year if the average grade gain 
was less than 1.5 years. 

Pre- and post-testing were conducted using a wide-range (i.e., uni- 
level) achievement test; the test's exact identity was a secret until 
the end of tie school year. Testing was conducted individually, rather 
than in group settings. The Coldbrook project adds another twist to 
the omnipresent problems of testing for performance contracts. There 
are no achievement tests normed on this type of student population. 
Whitecraft is working on a six-level test based on objectives provided 
by Alpha, but it will take several years to norm locally. 

Other evaluation devices included self<-concept tests. In May, 
questionnaires were used to solicit opinions on the project from parents^ 
teachers, and w^tudents. 

Whitecraft was very positive about the Alpha program, as was 
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everyone else on the district staff. Post-testing was not complete at 
the time of our site visit, but Nations Schools (September 1972) later 
reported that the average grade gains were 0.97 in math and 0.6 in read- 
ing. Beyond cognitive gains, there were very obvious gains in self-con- 
cept. Whitecraft said that teachers were initially opposed to the Alpha 
contract, but all except one were cooperating by the end of the school 
year. 

The Coldbrook project used Education Turnkey Systems Inc. for man- 
agement support. ETS wrote the contract between the district and Alpha, 
helped select achievement tests, and acted as a troubleshooter . White- 
craft said that management support was especially helpful during the 
trying first few months when materials were late in arriving and Alpha 
support was unstable. 

.,^NERAL IMPACTS OF CONTRACT LEARNING 

From the community standpoint, Henry Erb of Grand Rapids' Station 
WOTV reported that contract learning is not conspicuous among the many 
different educational programs underway in Grand Rapids. There has been 
little controversy about the contract learning programs. 

There seem to have been impacts on the manner in which the district 
is operated, however, and these are discussed in the following para- 
graphs . 

Impacts on Instruction 

The impacts of the contract learning programs on instruction have 
been evident. The district appears to be actively pursuing widespread 
use of systematic approaches to instruction, and is not restricted to 
those approaches offered by the three former performance contractors. 
At Ridgemoor Elementary School, for example, a Westinghouse program 
for individualized instruction. Project PLAN, is to be initiated in 
1972-73 for reading, math, science, and social studies. The program 
offers on-line diagnostic and prescriptive services via a computer ter- 
minal in the scuuol building. The cost per subject is about the same 
as for the other contract learning projects. One district administrator 
expressed misgivings about such a comprehensive program* He doubted 
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whether Individualized learning should necessarily equate with "individ- 
ually prescribed at-your-own-speed learning." He felt there should 
be more group learning, and this is one reason why contract learning 
for individually prescribed instruction may never have widespread appli- 
cation beyond basic skills. 

Central High, Ottawa Hills High, Union High, and Northeast Junior 
High have been planning programs for 1972--73 patterned after the learn- 
ing center model. It is interesting to contrast the first two programs, 
which will be quite different from each other. These differences reflect 
differences in the administrators and their students, faculty, and at- 
tendance areas. 

The Central High program is to be essentially remedial. Students 
with reading problems will be told they have not been doing well, and 
a rather bleak picture of their future in a literate society will be 
described but participation in the program will be voluntary. The design 
of the program will be influenced by observations made by Central High 
staff members inside and outside Grand Rapids. The Central High staff 
does not particularly regard this program as part of the contract learn- 
ing activities in Grand Rapids. 

Ottawa Hills High is a new school that draws from a white upper- 
class attendance area, but black students are bused in from poorer 
areas. Its program could not be remedial because most remedial students 
are ^rom the bused- in group, and administrators did not want the pro- 
gram to accent racial differences. Consequently, all students will be 
involved in the program, but scheduling is to be such as to decrease 
the awareness of differences. As of May 1972, the schc .1 was favoring 
an Alpha type of program, but no decision had been made. 

« Individualized instruction at least in reading and math is now 
P^bably more of a reality in Grand Rapids than before the institution 
of the contract learning programs. In many cases, it appears that 
teachers have escalated their perceptions of self-efficacy. They are 
becoming aware of a broader range of tools available to increase student 
achievement. Remedial reading teachers, the traditional foe of contract 
learning programs, have been investigating contractor's techniques. In 
almost all cases, the initiative for implementing new contract learning 
^ programs in 1971-72 came from teachers. 

ERIC 
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Notwlthstandlng the Project PLAN program. It appears that there 
has been a certain amount of contractor entrenchment in Grand Rapids, 
and that current contractors have the inside track for new programs. 
They are sufficiently aware of the needs in Grand Rapids to submit 
proposals that are more specifically tailored to Grand Rapids than those 
of their competitors. 

Impacts on District Administration 

There was some doubt among the district staff that the performance 
contract experience had a direct effect on the district's administrative 
skills and practices. It may be speculated that if there were such an 
impact, it would be eclipsed by the impact of the freewheeling manage- 
ment style of Phillip Runkle, the new school superintendent. There 
appears to be a possibly related shift toward decentralizing responsi- 
bility among building principals accompanied by low-keyed accountability. 
For the first time, principals are being given discretionary money 
(about §10 per student) and told to use their management ingenuity. 

David Thompson of the Grana Rapids Education Association feels 
that the performance contracting experience has increased the awareness 
of the board and staff of the value of in-service training as a means 
of getting teachers involved in their work arid as a means of imparting 
information. He suggested that some administrators were surprised at 
the many cases of sustair i high performance — as if the upper level of 
teacher performance had been underestimated in previous years. Thompson 
has been critical of the district's traditional training programs. The 
contract learning programs » on the other hand, have all been supported 
by professional consultation, and the contractors have taken this train- 
ing function very seriously. A hopeful portent for the future is that 
the district appointed a director for in-service training last year. 
Joan Webster, Coordinator of Contract Learning, says this is a manifes- 
tation of the growing realization in the district offices that "account- 
ability is a two-way street," and that the district is obliged to engage 
heavily in training and other instruction-supportive activities. 

Joan Webster, who was the on-site director of the OEO experiment 
in 1972-71, became the district's Coordinator of Contract Learning for 
1971-72. Two of the district's seven contracts: the Coldbrook contract 
and the Alternative Education Center contract, however, dso had indi- 
vidual project directors. 
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Impact on Resources 

Grand Rapids' Increased Involvement In contract learning has meant 
very little In regard to shifts In resource allocaf.ion. Except for 
South Middle School (CMES) , no contract learning program required an 
Increase in staffing. In some cases the staffing decreased. When a 
contract program replaced a conventional remedial reading program, the 
remedial reading teacher was usually assigned to the contract program. 

Most of the contractor fees were absorbed by a Title I windfall. 
Because Title I eligibility reflects welfare rolls, the recent General 
Motors strike increased last year's Title I funding in Grand Rapids from 
$1 million to $1.3 million. Wallace Norgrove, the district's Director 
of Special Programs, was apprehensive that the Title I funding might 
decline 10 percent for the 1972-73 school year. 

The district's funds were also bolstered by a sort of "reverse" per- 
formance contract with the state of Michigan. On the basis ^f the 1971 
Michigan Assessment of Education, it was determined that 27.6 percent 
of Grand Rapids' students were below the fifteenth percentile state-wide 
in reading and math. This qualified Grand Rapids for an extra million 
in "State of Michigan Section 3" funding for three years. Each student 
must achieve a certain threshold of performance in reading and math in 
order to keep the funding level constant. Ocherwlse, funding will be 
decreased proportionally. This funding arrangement is a concept of State 
Superintendent Porter, who is a supporter of the performance contracting 
concept. 

In all programs, teachers' salaries came from regular operating 
funds. Aides at Coldbrook School were paid from Title III funds; aides 
In all other programs were paid from Title I money. The contractor fee 
for Coldbrook was also fro* Title III, and fees for the other Alpha con- 
tracts were from Title I. Fees for the CMES and Learning Unlimited pro- 
grams were split almost evenly between Title I and State of Michigan 
Section 3 monies. 



Impact on Evaluation and Research 

Evaluation of the contract learning programs hae been absorbed into 
the district's regular testing and evaluation program. The 1971 spring 



t*^8t results are used In place of fall pre-testlng where possible, even 
for the performance contracts. 

The district has formally encouraged local research Into aspects 
of performance contracting. Three studies, all conducted during the 
1970-71 school year, have been reported. 

Mary Edmond, an assistant principal at South Middle School, con- 
du'-ted a study to assess changes in student attitudes in that school, 
Tho results to date have been inconclusive, but the data will probably 
be reanalyzed. 

Gordon DeBlaey and Henry Holstege, both of Calvin College, and 
James D. Jone^ , of Western Michigan Univ^.rsity, conducted a study of 
the perception of performance contracting oy nonpa^^tlcipaf ipg t^acl.ers 
in Grand Rapids. About half of the 100 teachers who responded to the 
survey failed to rate performance contracting either positively or nega- 
tively, but almost all of the remainuer were positive. Surprisingly, 
younger teachers were less favorable than older teachers. Lower ele- 
mentary teachers generally felt that contracted programs of the type 
in operation in Grand Rapids should be restricted to low achievers, 
whereas upper elementary and junior high school teachers felt that all 
students should partf-ipate in the programs. 

While she was Project Director for the GEO experiment in Grand 
Rapids, Joan Webster developed and implemented a format for cost-effec- 
tiveness analysis, using data from the 19 70-71 contracted and traditional 
remedial programs. School officials credit this study with providing 
valuable policy assistance in shaping remedial program priorities. 

The Center for Educational Studies, a Joint venture of the district 
and Western Michigan Uni-^ersity, has sponsored two conferences on re- 
search In performance contracting. James Bosco, director of the center, 
told us that the GEO evaluation has had a devastating effect on the 
center's ability to interest researchers in this and related problems. 
In effect, at this point there is no research on performance contracting 
in Grand Rapids. 

View from the Union 

The teachers' union (Grand Rapids Education Association) has warmed 
toward contract learning. It was disturbed the first year (1970-71) 
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about Alpha's alleged surreptitious teacher bonuses, but this problem 
has now been resolved. If the staff wants to have a learning center, 
and if there is no violation of the master contract between the union 
and the district, the union states that it is satisfied. All contracts 
for 1971-72 included the phrase "Terms of the Master Agreement between 
the Grand Rapids Edu .ation Association and Grand Rapids School District 
are not negated by anything in this contract and shall remain in full 
force and effect." 

GREA officials try to meet with teaching staffs that are consider- 
ing establishing a learning center to explain that there may be many 
operating differences, such as differential staffing and less regular 
scheduling of classes. According; to GREA*s Executive Director, David 
L. Thompson, when only a few teachers are violently opposed to the learn- 
ing center the union stance is to suggest that they transfer (with union 
help) to another school where they will be happier and probably more 
effective. 

SUMMARY 

The shift from performance payment to flat-rate and/or turnkey 
seems to have been primarily motivated by a desire to obtain greater 
flexibility in programming students. In some cases, it also costs less 
and permits wider implementation of the elements introduced by performance 
contracting, accommodating more students at more grade levels. Teachers 
also want more control of instruction. 

There are disadvantages, of course. Some teachers »Te not efficient 
in the special sort of management responsibilities that ere required. 
Contractor support is spread more thinly. Also, there is less contractual 
authority, and contractors tend to be less responsive in supplying ma- 
terials and services on time. In some instances during 1971-72, Grand 
RapiQS threatened to withhold payment in order to get contractors to act. 
In future contracts, the school district plans to develop detailed timing 
schedules with penalty clauses. For the kinds of techniques introduced 
in Grand Rapids, it appercb that one year is not sufficient exposure for 
the district to carry on alone. It seems desirable to have at least one 
transition year during which there is strong contractor consultation 
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support. At least. Grand Rapids feels that the 1971-72 year in which 
there was substantial contractor Involvement in its programs was a de- 
sirable step between the 19 70-71 prograiLS and the goal of school ope- 
rated programs . 

In Grand Rapids, the concept of performance contracting has become 
Indelibly identified with individualized instruction. We asked two school 
administrators whether Grand Rapids might one day again resort to per- 
formance contracting in order to introduce an Innovation into the district, 
but neither could conceive of circumstances that would tempt them to con- 
sider such a recommendation. It was not clear whether they were negative 
toward performance contracting or irtiether they felt its potential in 
Grand Rapids had been reached. What is clear, however, was summed up 
in the coimnent that "performance contracting will probably go away, but 
the residual benefits will remain. The teachers will never be the same." 
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VII. CONCLUSIONS 

INTRODUCTION 

In general, our review of activities In the five school districts 
that we studied in depth In 1970-71 reinforced the conclusions of that 
study. We will review the inore significant of those conclusions and 
add some further comments In light of the 1971-72 activities. 

Performance contracting In the five districts discussed In this 
report seems to have brou^t about changes In the content and methods 
of Instruction, measurement and evaluation, and management. In anothe 
area, the pattern of use of school resources, performance contracting 
may not be having a lasting effect. In still another area, the effec- 
tiveness of Instruction, definitive data are scarce but gains achieved 
in some programs appear superior to gains achieved in conventional 
remedial programs, while in other cases the results are about the same 

CONTENT AND METHOD OF INSTRUCTION 

o In each of the five districts, the use of a diagnostic- 
prescriptive approach to individualization of Instruction 
has continued and, in some cases, has spread to more 
schools and students. In one case. Individualization has 
been applied to additional areas of the curriculum. 

o The performance contracting technique can be a mech- 
anism for ensuring a more uniform approach to instruction 
among classrooms. Without direct contact with a con- 
tractor under the pressure of performance-related pay- 
ments teachers are likely to adjust the program to 
their individual teaching styles and more diversity 
among classrooms will result* 

o There has been Increased emphasis on providing a wide 
variety of self-instructional materials and equipment 
appropriate both to the students* social maturity and 
to their level of cognitive development. 
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o The performance contracting technique is still largely limited 
to the basic skills of reading and mathematics. 

RESOURCE USE 

The use of paraprofesslonals In the classroom and other features 
of the 1970-71 programs raised the hope that performance contracting 
might lead to a reduction In the cost of remedial programs. The suc- 
ceeding year's experience did not strengthen this hope, however. With 
one exception, programs used about the same resource support as they 
had the year before or even greater support was provided. 

EFFECTIVENESS OF INSTRUCTION 

We have only limited data on achievement gains during 1971-72. 
In Gilroy, reading reachers using the WLC approach were markedly suc- 
cessful In Improving their students' scores. In the other districts, 
the results of the performance contracting programs or their progeny 
are mixed. In Gary the 22-day teacher strike led to cancellation of 
the post- test, so no gain scores are available. In Texarkana the gains 
were uniformly less than the objectives, for example. Instead of 75 
percent of the students In the labs making one grade level gain or 
better, only 38 percent made one grade level gain or better In reading 
comprehension and only 28 percent In arithmetic. In Norfolk, the pro- 
gram group In most but not all cases did not show gains significantly 
different from the control group. In Grand R^'nids the results were 
mixed; one program had gains of about two achievement years and the 
others were in the 0.5 to 1.2 range. In Gilroy In a program started 
in 1971-72 only two of the 29 students achieved the target gain of nine 
months' growth. The students who had been In the 1970-71 program but 
were not In such a program In 1971-72 did exceptionally well on the 
third-grade reading test. The fourth-grade results were no better than 
expected for those who had participated In the program but nonpartlcl- 
pants made substantial gains. Whether these results were due to curric- 
ulum improvements can't be determined. 
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MEASUREMENT AND EVALUATION 

Because performance contracting ties contractor payment to student 
test results. It focuses on testing and evaluation. This has had sev- 
eral effects: 

o Teachers and administrators have become more supportive 
of efforts to emphasize student learning as a measure 
of the effectiveness of Instruction. 

o There has been Increased emphasis In the research com- 
munity on Improving measuring Instruments. For example, 
(1) a number of organizations have been working to de- 
velop reliable learning-mastery tests, and (2) additional 
work has gone into refining the interpretation of 
standardized achievement tests. 

o There has been increasing concern at all levels of res- 
ponsibility within the schools with producing useful 
and Informative evaluations. 

MANAGEMENT 

Through the performance contracting experience, school administrators 
became Intensively Involved with the course of the project— from contrac- 
tual arrangements to the classrooms. This has had the effect of edu- 
cating and sensitizing administrators to problems and possibilities in 
a number of areas: 

o The development and implementation of new approaches 

to instruction, 
o Teacher training. 

o Test selection and evaluation design. 

o Union pressures. 

o Federal funding arrangements. 



